Visible-Infrared Image Alignment for UAVs: Benchmark and New Baseline

With the extensive use of multisensors in uncrewed aerial vehicles (UAVs), multimodality information processing has become the research focus. In academic research pertaining to object detection and tracking tasks in UAVs, researchers often align visible-infrared image pairs as a preprocessing step. However, in actual tasks, the dual-modality image pair acquired by UAVs is unaligned, which significantly limits the application of downstream tasks. At present, there are no publicly available multimodality image alignment datasets for UAVs. In this article, we present a large-scale benchmark for the dual-modality image alignment task in UAVs, including 81000 training image pairs and 15000 testing image pairs. Meanwhile, we propose a transformer-based dual-modality image alignment network as the baseline for this benchmark. First, the algorithm extracts multiscale features for image representation to address unaligned image pairs with varying resolutions. Second, a transformer-based alignment network is proposed to improve the fusion of features from heterogeneous modalities. Finally, deformable attention is adopted to alleviate the problem of memory explosion. Numerous experiments on this dual-modality image alignment benchmark are conducted to demonstrate the effectiveness of our algorithm. Source codes are available at https://github.com/gaozhinanjiu/UAVmatch.