Scale-Aware Attention and Multi-Modal Prompt Learning With Fusion Adapter for RGBT Tracking

Fusing visible (RGB) and thermal (T) images for RGBT tracking has received growing interest in the field of computer vision. However, how to improve the robustness of the tracker to target scale variety, effectively apply visual prompts to multimodal tracking tasks, and enhance the multimodal fusion effectiveness are still urgent challenges in the field of RGBT tracking. To this purpose, this work proposes an RGBT tracking framework integrating scale-aware dilation attention, multimodal prompt interaction learning, and cross- fusion adapter, named MPANet. Firstly, a scale-aware dilation attention (SADA) module is put forward to enhance the flexibility of the tracker in the presence of target scale variations by embedding convolutions with different dilation rates into the self-attention. Subsequently, a multimodal prompt interaction learning (MPIL) module is constructed, which combines global token adaptive attention and spatial attention to efficiently learn visual prompts from different modalities and achieve intermodal prompt interactions. Finally, a cross-fusion adapter (CFA) is developed to facilitate the adaptability of the network to different modalities in the process of multimodal information fusion through the adapter mechanism. Extensive experiments on public RGBT benchmark tracking datasets such as GTOT, RGBT234, LasHeR and VTUAV demonstrate that the proposed method outperforms existing advanced trackers and achieves state-of-the-art performance.