MSFNET: Multi-Stage Fusion Network for Semantic Segmentation of Fine-Resolution Remote Sensing Data

This work proposes a Multi-Stage Fusion Network (MSFNet) for semantic segmentation of fine-resolution remote sensing data by exploiting a multi -stage transformer architec-ture. The proposed MSFNet fuses information of different scales and modalities using a multi-stage scheme based on cross-attention mechanism. More specifically, the proposed MSFNet is composed of two Multi-Level Transformers (ML-Trans), one Crossmodal Fusion Transformer (CFTrans) and one Global-Context Augmented Transformer (GCATrans). DMLTrans and CFTrans are designed to fuse features in dif-ferent levels in each modality and high-level crossmodal ab-stract features, respectively, whereas GCATrans enhances the fusion feature of the main modal. Capitalizing on MSFNet, this work demonstrates the fusion of red-green-blue (RGB) remote sensing images and digital surface model (DSM) data. Extensive experiments on large-scale fine-resolution remote sensing data sets, namely the ISPRS Vaihingen, confirm the excellent performance of the proposed architecture as compared to conventional multimodal methods.