Improving soybean yield prediction by integrating UAV nadir and cross-circling oblique imaging

High-throughput estimation of soybean yield using unmanned aerial vehicle (UAV) imagery can help improve the efficiency of soybean breeding. Previous studies have mainly focused on the extraction of vegetation indices and texture features from two-dimensional(2D) orthophotos to construct empirical models of yield, lacking spatial structure information of crops. Therefore, UAV cross-circling oblique (CCO) photography combined with SfM-MVS algorithm was used to reconstruct three-dimensional(3D) soybean canopy structure. Then canopy 3D related phenotypic features are extracted and combined with features from RGB nadir and multispectral images to analyze the capability of different modal data fusion on soybean yield prediction. In addition, Shapley value was used to evaluate the importance of features across different machine learning models. Based on the Shapley value, a bagging-stacking ensemble learning framework was developed using Lasso, Random Forest (RF), Ridge Regression (RR), and XGBoost as base learners for yield prediction. The performance of the traditional stacking method was evaluated and compared with weighted average methods such as Bayesian Model Averaging (BMA) and Entropy Weighted Average (EWA) as meta-learners in the ensemble framework. The results show that 3D canopy structure of soybean can be obtained from UAV CCO photography. The inclusion of 3D structural features can improve the accuracy of yield estimation. Among different modal data combinations, the highest estimation accuracy was achieved when combining RGB nadir features with CCO 3D features. The performance of the above base learners was improved by 8.8%, 3.5%, 7.1%, and 8.0% respectively when using the Shapley value method. The accuracy of yield prediction applied on independent dataset of year 2023 was further calculated by using the bagging-stacking ensemble learning method. When using BMA as the meta-learner, the best performance is obtained with an R2 of 0.7. Therefore, UAV CCO photography with SfM-MVS algorithm provides a new approach to obtain high-quality point clouds of the crop canopy at low cost. UAV multimodal data combined with ensemble learning models allow accurate estimation of yield prediction in breeding materials of soybeans at the plot scale.