MSFR‐Net: Multi‐modality and single‐modality feature recalibration network for brain tumor segmentation

Abstract Background Accurate and automated brain tumor segmentation from multi‐modality MR images plays a significant role in tumor treatment. However, the existing approaches mainly focus on the fusion of multi‐modality while ignoring the correlation between single‐modality and tumor subcomponents. For example, T2‐weighted images show good visualization of edema, and T1‐contrast images have a good contrast between enhancing tumor core and necrosis. In the actual clinical process, professional physicians also label tumors according to these characteristics. We design a method for brain tumors segmentation that utilizes both multi‐modality fusion and single‐modality characteristics. Methods A multi‐modality and single‐modality feature recalibration network (MSFR‐Net) is proposed for brain tumor segmentation from MR images. Specifically, multi‐modality information and single‐modality information are assigned to independent pathways. Multi‐modality network explicitly learns the relationship between all modalities and all tumor sub‐components. Single‐modality network learns the relationship between single‐modality and its highly correlated tumor subcomponents. Then, a dual recalibration module (DRM) is designed to connect the parallel single‐modality network and multi‐modality network at multiple stages. The function of the DRM is to unify the two types of features into the same feature space. Results Experiments on BraTS 2015 dataset and BraTS 2018 dataset show that the proposed method is competitive and superior to other state‐of‐the‐art methods. The proposed method achieved the segmentation results with Dice coefficients of 0.86 and Hausdorff distance of 4.82 on BraTS 2018 dataset, with dice coefficients of 0.80, positive predictive value of 0.76, and sensitivity of 0.78 on BraTS 2015 dataset. Conclusions This work combines the manual labeling process of doctors and introduces the correlation between single‐modality and the tumor subcomponents into the segmentation network. The method improves the segmentation performance of brain tumors and can be applied in the clinical practice. The code of the proposed method is available at: https://github.com/xiangQAQ/MSFR‐Net .