Disentangle and Then Fuse: A Cross-Modal Network for Synthesizing Gadolinium-Enhanced Brain MR Images

Despite the widespread use of gadolinium-based contrast agents in clinical MRI examinations due to their significant advantages in structural localization and tumor identification, there is a risk of brain deposition and nephrogenic systemic fibrosis. Cross-modal image synthesis methods offer a new alternative, yet lesion synthesis remains challenging. On one hand, brain lesions vary significantly in location, shape, and size. On the other hand, the high background ratio associated with brain lesions makes their synthesis more difficult. To address these issues, we first introduce a Multi-Objective Local Perception Module (M-OLPM), which utilizes edge generation and lesion segmentation tasks to prioritize local lesions from the disentangled local perceptual feature subspaces. To better extend to multi-objective local perception, we propose a ‘Disentangle and Then Fuse’ learning strategy, including a Feature Disentanglement Module (FDM) and a Global Fusion Module (GFM). The FDM decouples multimodal deep features into low-frequency semantic features and high-frequency edge features, alleviating feature conflicts from weakly related perception tasks. To enhance feature interaction among multiple perception tasks, the GFM progressively integrates these local perceptual features and underlying detail features through an attention mechanism, further refining the global image quality. Evaluated on the publicly available BRaTS2020, BRaTS2021 datasets, and the private HPPH dataset, our method significantly outperforms the existing technology in both visual and quantitative assessments of gadolinium-enhanced MRI images in global and localized lesion areas, providing a safe alternative to gadolinium enhancement. The source code is publicly available at https://github.com/zengyangche/DTF-Net.