Missing Data Imputation via Conditional Generator and Correlation Learning for Multimodal Brain Tumor Segmentation

• A conditional generator is designed to generate the missing modality with correlation constraint among the modalities. • A multi-source correlation network is introduced to learn the latent multi-source correlation among modalities. • A brain tumor segmentation network via multi-source correlation to conditionally generate the missing modality is proposed. • The experimental results demonstrate the effectiveness of our proposed method. Brain tumor is one of the most high-risk cancers which causes the 5-year survival rate of only about 36%. Accurate diagnosis of brain tumor is critical for the treatment planning. However, it’s common to missing one modality in clinical scenarios. In this paper, we propose a novel brain tumor segmentation network to impute the missing data. The proposed network consists of a conditional generator, a multi-source correlation network and a segmentation network. To impute the missing data, we propose to use a conditional generator to generate the missing modality under the condition of the available modalities. As the multi MR modalities have a strong relationship in tumor regions, we design a multi-source correlation network to learn the multi-source correlation. On the one hand, the multi-source correlation network can help the conditional generator to generate the missing modality which should keep the consistent correlation with the available modalities. On the other hand, it can guide the segmentation network to learn the correlated feature representations to improve the segmentation performance. The experiments evaluated on BraTS 2018 dataset demonstrate the superior performance of the proposed method when compared with the state-of-the-art methods.