Integrating deep learning and data fusion for enhanced oranges soluble solids content prediction using machine vision and Vis/NIR spectroscopy

The visible/near infrared (Vis/NIR) spectrum will become distorted due to variations in sample color, thereby reducing the prediction accuracy of fruit composition. In this study, we aimed to develop a deep learning model with color correction capability to predict oranges soluble solids content (SSC) based on multi-source data fusion. Initially, a machine vision and Vis/NIR spectroscopy online acquisition device was designed to collect and analyze color images and transmission spectra. Subsequently, data fusion methods were proposed for color features and spectral data. Finally, color-correction one-dimensional convolutional neural network (1D-CNN) models base on multi-source data were constructed. The results showed that, the RMSEP of optimal color-correction model was decreased by 36.4 % and 16.1 % compared to partial least squares model and conventional 1D-CNN model, respectively. The multi-source data fusion of machine vision and Vis/NIR spectroscopy has the potential to improve the accuracy of food composition prediction.