UDFusion: Uncertainty-Driven Fusion for Multimodal Remote Sensing Image Classification with Dynamic Sampling Strategy

The task of land cover classification in remote sensing imagery is a crucial research topic in the field of remote sensing. Deep learning methods, driven by large datasets, have made significant progress in this task. In current research on land cover classification, fixed image patch selection strategies overlook the correlations between pixels. Irrelevant information or information from other categories contained in the patches can actually interfere with the network’s learning of the correct category. Meanwhile, the continuous expansion of data sources has made it possible to integrate information from different modalities. However, the effectiveness of the interaction between different modal data can have a huge impact on the network’s performance, and traditional fusion methods still struggle to demonstrate sufficient reliability. To address these issues, we introduce a wavelet transform-based variable patch selection method and an uncertainty-aware decision-level fusion method, which provide more positive information to the network and promote interaction between modalities, thereby enhancing the performance of multimodal networks. We will validate the performance of the designed network on benchmark dataset.