Multimodal Brain Tumor Segmentation Using Contrastive Learning Based Feature Comparison with Monomodal Normal Brain Images

Many deep learning (DL) based methods for brain tumor segmentation have been proposed. Most of them put emphasis on elaborating deep network's internal structure to enhance the capacity of learning tumor-related features, while other valuable related information, such as normal brain appearance, is often ignored. Inspired by the fact that radiologists are often trained to compare with normal tissues when identifying tumor regions, in this paper, we propose a novel brain tumor segmentation framework by adopting normal brain images as reference to compare with tumor brain images in the learned feature space. In this way, tumor-related features can be highlighted and enhanced for accurate tumor segmentation. Considering that the routine tumor brain images are multimodal while the normal brain images are often monomodal, a new contrastive learning based feature comparison module is proposed to solve incomparable issue between features learned from multimodal and monomodal images. In the experiments, both in-house and public (BraTS2019) multimodal tumor brain image datasets are used to evaluate our proposed framework, demonstrating better performance compared to the state-of-the-art methods in terms of Dice score, sensitivity, and Hausdorff distance. Code: https://github.com/hbliu98/CLFC-Brain-Tumor-Segmentation .