A deep learning approach combining instance and semantic segmentation to identify diseases and pests of coffee leaves from in-field images

The automated diagnosis of pests and diseases that affect coffee crops is an important issue for coffee farmers. Conventional methods of computer vision and pattern recognition present limitations to tackle such challenging problems. However, in the last few years, there is a growing interest in deep learning, especially in the detection/recognition of biotic stresses from in-field images of plants acquired by smartphones, since they are affected by lighting variations, complex backgrounds, image noise, and so on. In this work, we propose an integrated framework by using different convolutional neural networks (CNN) to automate detection/recognition of lesions from in-field images collected via smartphone containing part of the coffee tree. In the first stage, we use a Mask R-CNN network for instance segmentation; in the second stage the UNet and PSPNet networks for semantic segmentation and finally, in the third stage, a ResNet for classification. For the Mask R-CNN network, we obtained a precision of 73.90% and a recall of 71.90% in the instance segmentation task. For the UNet and PSPNet networks, we obtained a mean intersection over union of 94.25% and 93.54%, respectively. The results are promising and indicate suitability to implement the entire framework in an embedded mobile platform to be used in the real world.