Style transfer in conditional GANs for cross-modality synthesis of brain magnetic resonance images

Magnetic resonance imaging (MRI) has become one of the most standardized and widely used neuroimaging protocols in the detection and diagnosis of neurodegenerative diseases. In clinical scenarios, multi-modality MR images can provide more comprehensive information than single modality images. However, high-quality multi-modality MR images can be difficult to obtain in the actual diagnostic process due to various uncertainties. Efficient methods of modality complement and synthesis have aroused increasing attention in the research community. In this article, style transfer is introduced into conditional generative adversarial networks (cGAN) architecture. A cGAN model with hierarchical feature mapping and fusion (ST-cGAN) is proposed to address the cross-modality synthesis of MR images. In order to surmount the sole focus on the pixel-wise similarity as most cGAN-based methods do, the proposed ST-cGAN takes advantage of the style information and applies it to the synthetic image's content structure. Taking images of two modalities as conditional input, ST-cGAN extracts different levels of style features and integrates them with the content features to form the style-enhanced synthetic image. Furthermore, the proposed model is made robust to random noise by adding noise input to the generator. A comprehensive analysis is performed by comparing the proposed ST-cGAN with other state-of-the-art baselines based on four representative evaluation metrics. The experimental results on the IXI (Information eXtraction from Images) dataset verify the validity of the ST-cGAN from different evaluation perspectives.