高光谱成像
多光谱图像
计算机科学
人工智能
模式识别(心理学)
卷积神经网络
特征提取
计算机视觉
作者
Qing Ma,Junjun Jiang,Xianming Liu,Jiayi Ma
标识
DOI:10.1016/j.inffus.2023.102148
摘要
Hyperspectral and multispectral (HS–MS) image fusion aims to reconstruct high-resolution hyperspectral images from low-resolution hyperspectral images and high-resolution multispectral images. While convolutional neural networks have been widely used for HS–MS fusion, their potential is curtailed due to the limited receptive field of each neuron, resulting in inadequate long-range modeling capabilities. Although several Transformer-based HS–MS fusion methods have been proposed, most of them often fail to fully integrate and coordinate the data from the two modalities (i.e., hyperspectral images and multispectral images). Such ineffective interactions significantly compromise the quality of the reconstructed hyperspectral images. In this paper, we introduce a novel reciprocal fusion strategy called the dual cross Transformer-based fusion (DCTransformer) for HS–MS fusion. The model excels in capturing the interplay between different modalities by utilizing directional pairwise multi-head cross-attention, which facilitates the transfer of information between modalities. Additionally, we incorporate a Swin Transformer block post cross-attention to enhance the self-attention within the context. Extensive experiments show that our DCTransformer performs favorably against other recent works on both simulation HSI datasets and real HSI datasets. The source code and pre-trained models are availabe at https://github.com/qingma2016/DCTransformer.
科研通智能强力驱动
Strongly Powered by AbleSci AI