Detection of citrus black spot disease and ripeness level in orange fruit using learning-to-augment incorporated deep networks

Fruit infected by pests or diseases and fruit harvests with different levels of ripeness cause a lack of marketability, decrease in economic value, and increase in crop waste. In this study, we propose a robust and generalized deep convolutional neural network (CNN) model via fine-tuning the pre-trained models for detecting black spot disease and ripeness levels in orange fruit. A dataset containing 1896 confirmed orange images in the farm in four classes (unripe, half-ripe, ripe, and infected with black spot disease) was used. In order to prevent overfitting and increase the robustness and generalizability of the model, instead of using fundamental data augmentation techniques, a novel learning-to-augment strategy that creates new data using noisy and restored images was employed. Controllers using the Bayesian optimization algorithm were utilized to select the optimal noise parameters of Gaussian, speckle, Poisson, and salt-and-pepper noise to generate new noisy images. A convolutional autoencoder model was developed to produce newly restored images affected by optimized noise density. The dataset augmented by the best policies of the learning-to-augment strategy was used to fine-tune several pre-trained models (GoogleNet, ResNet18, ResNet50, ShuffleNet, MobileNetv2, and DenseNet201). The results showed that the learning-to-augment strategy for the fine-tuned ResNet50 achieved the best performance with 99.5% accuracy, and 100% F-measure by assigning images infected with black spot disease as the positive class. The proposed automatic disease and fruit quality monitoring technique can be also used for the detection of other diseases in agriculture and forestry. • Providing a stabilized model for grading ripeness level of orange fruit. • Augmenting data by the best policies of optimal noise parameters. • Preventing the overfitting by generating new data using the proposed autoencoder. • Achieving better performance as compared to state-of-the-arts. • Getting 99.5% accuracy for pest detection and classifying orange fruit quality.