Multimodal deep learning with hyperspectral imaging for accurate origin classification of wolfberries

Accurate classification of wolfberry geographical origin is essential for assessing its nutritional and medicinal properties. A multimodal convolutional neural network (MTCNN) with a cross-attention mechanism was proposed to effectively fuse spectral and image features, achieving a test accuracy of 99.88 %. Traditional feature extraction methods such as and principal component analysis (PCA) were combined with support vector machine (SVM) and k-nearest neighbor (KNN) classifiers. The SVM model using gradient boosting decision tree (GBDT) -extracted spectral features and fused image data achieved the highest accuracy of 96.68 %. The comparison results highlight the superior performance of multimodal deep learning over conventional methods, demonstrating its potential for robust agricultural product traceability, authentication, and quality assurance. Beyond achieving higher accuracy, our model improves upon previous architectures by reducing computational complexity through the use of a simplified attention mechanism and enhancing interpretability, making the model more efficient and accessible for practical applications.