MIFTr: Efficient Universal Image Matching With Modality-Invariant Feature Transformer

Image matching is a fundamental process in intelligent image processing, especially for the wide application of multi-source remote sensing technologies, where invariant feature extraction is a core technology. Deep learning-based matching networks have emerged in recent years due to their powerful perception capabilities and parallel computing efficiency. However, most of them are designed for optical or a specific modality. Only recently general cross-modal matching has gained attention, but the prevailing approaches mainly rely on fine-tuning existing models or utilizing frozen weights from large pre-trained models, with little focus on dedicated models tailored for cross-modal scenarios. In this paper, we propose Modality-Invariant Feature Transformer (MIFTr), a lightweight image matching model dedicated to achieving universal cross-modal capability. The model innovatively introduces a rotational multi-branch backbone, and incorporates the latest MambaVision Mixer to construct a Symmetric Cross-Vision-Mamba (CrossVM) module as the core interactive feature encoder. With sufficient pretraining, the model handles general types of image modalities with processing time within 40ms per 5122sample on RTX4090. Through extensive comparative experiments on several complex cross-modal datasets, the proposed MIFTr demonstrates consistently superior performance on each modality, while maintaining a minimal model size and exceptional computational efficiency. The codes and experimental data will be made publicly available athttps://github.com/MrPingQi/MIFTr.