计算机科学
高光谱成像
人工智能
强化学习
激光雷达
融合
马尔可夫链
传感器融合
马尔可夫过程
模式识别(心理学)
机器学习
遥感
语言学
哲学
统计
数学
地质学
作者
Haoyu Wang,Yuhu Cheng,Xiaomin Liu,Xuesong Wang
标识
DOI:10.1109/tmm.2024.3360717
摘要
Hyperspectral images (HSIs) and light detection and ranging (LiDAR) are two critical and frequently used types of remote sensing data, each containing rich spectral and elevation information. Fusing HSI and LiDAR can exploit the complementary properties of the two modalities for ground object classification. The performance of existing fusion classification methods is often limited by the difficulty of adapting feature extraction operators to complex spatial distributions, and the correlation and specificity between different modalities are not reasonably exploited. Therefore, the reinforcement learningbased markov edge decoupled fusion network (MEDFN) is proposed. This network can intelligently compose graphs based on different modal characteristics and tasks to adapt to complex spatial distributions; it can also suppress noise to complete fusion classification while fully utilizing complementary information of different modalities. First, a reinforcement learning-based graph construction subnetwork (RLGN) is proposed to learn a twomodal graph construction strategy suitable for classification tasks by transforming regular multimodal data into irregular graph data. Second, a multimodal edge attention module (MEAM) is proposed to extract edge features between spatial neighboring nodes and model the importance of each node, thereby capturing the spatial topology information encompassed in the multimodal data. Finally, the decoupled multimodal fusion module (DMFM) is proposed to decouple multimodal features into shared and unshared parts and enhance the model's ability to distinguish features by targeting the modal-shared feature between modalities and modal-specific feature. The experimental results based on three well-known HSI and LiDAR datasets demonstrate the effectiveness of the proposed MEDFN in fusion classification tasks.
科研通智能强力驱动
Strongly Powered by AbleSci AI