M3Net: A multi-model, multi-size, and multi-view deep neural network for brain magnetic resonance image segmentation

Abstract Segmentation of the brain into gray matter, white matter, and cerebrospinal fluid (CSF) using magnetic resonance (MR) imaging plays a fundamental role in neuroimaging research and clinical settings. Due to the complexity of brain anatomy, low image quality, and insufficient training data, both traditional and deep learning segmentation methods have a limited performance. In this paper, we propose a multi-model, multi-size and multi-view deep neural network (M3Net) for brain MR image segmentation, which uses three identical modules to segment transaxial, coronal, and sagittal MR slices, respectively. Each module consists of multi-size U-Nets and multi-size back propagation neural networks. It also uses a probabilistic atlas to explore brain anatomy and a convolutional auto-encoder (CAE) to restore MR images. The proposed M3Net model was evaluated against widely used segmentation methods on both synthetic and clinical studies. Our results suggest that the proposed model is able to segment Brain MR Images more accurately.