Complementary deep learning and chemometrics: A case of pear fruit centroid detection and spectral model application for fruit spectral image processing

A novel case of combining deep learning and chemometrics for spectral image processing is presented. The case involved the application of deep transfer learning for detecting and locating the fruit centroid to extract pixels for spectral model development and application. The selected fruit case involved a non-symmetrical fruit pear where the interesting area for spectral model application is not the centroid of the whole fruit unlike fruit such as apples but the centroid of the belly part of the pear fruit. Hence, the task of object detection is replaced with the task of symmetrical region (fruit belly) detection on the pear fruit such that the spectral model can be applied in the centroid pixels of the symmetrical region. For spectral modelling, the latent variables based regression technique called partial least-square (PLS) regression was used. For spectral modelling, PLS was preferred over deep learning as there was a low number of samples points to train a deep spectral model. The deep transfer learning allowed 100 % correct detection of the pear fruit belly part with the intersection over union score of 0.82. Furthermore, the RMSEP = 0.77 % was attained with the PLS modelling to predict dry matter. The presented approach can support the wide application of spectral imaging for fresh fruit analysis, particularly when imaging is performed simultaneously on multiple objects and the objects are non-symmetrical in shape.