MSAIF-Net: A Multistage Spatial Attention-Based Invertible Fusion Network for MR Images

In recent years, multi-modal medical image fusion has drawn increasing attention, aiming to provide comprehensive information for image understanding and clinical applications. With the development of deep learning-based models, great success has been achieved in multi-modal image fusion. However, most current models still suffer from limited performance and feature loss during fusion. The lack of consensus on evaluation criteria further compromises their clinical values. In this paper, to deal with aforementioned concerns, a novel multi-stage spatial attention based invertible fusion network (MSAIF-Net) is proposed for multi-modal MR images with improved feature learning and fusion ability. More specifically, it introduces the feature extraction module to exploit the foreground and background of feature maps and proposes a multi-attention module to aggregate features among different stages. Moreover, an invertible neural network is designed for feature fusion, so as to better preserve the complementary information of different modalities. In addition to comparing their unsupervised evaluation metrics, a high-level task, segmentation task, is adopted to test the superiority of fused images. Extensive experiments on different datasets validate the effectiveness and generalization of our proposed method compared with other state-of-the-art image fusion approaches in both supervised and unsupervised metrics.