Real-time MRI-ultrasound image translation under limited paired data using a physically motivated conditional GAN

Abstract Objective . Magnetic resonance imaging (MRI) and ultrasound (US) provide complementary anatomical and intraoperative information, yet their large appearance discrepancy makes cross-modality synthesis challenging. This study proposes a fast and physically consistent bidirectional MRI-US translation framework based on a conditional generative adversarial network (GAN). Approach . The generator adopts a VGG19-informed U-Net with residual bottleneck blocks and a self-attention module to capture long-range anatomical dependencies, while a multi-scale PatchGAN discriminator with spectral normalization improves texture realism and training stability. The model is optimized using a composite objective including least-squares adversarial, pixel-wise L1, and perceptual losses. To address limited paired data and enhance 3D consistency, a random slicing augmentation strategy is introduced to generate diverse oblique 2D slices from 3D volumes. Main results . Experiments on the RESECT (brain) and μ -RegPro (prostate) datasets demonstrate that the proposed method outperforms state-of-the-art convolutional neural network-, GAN-, and diffusion-based approaches in perceptual realism (FID/LPIPS), structural fidelity, and ultrasound speckle statistics equivalent number of looks. The proposed framework achieves millisecond-level inference (approximately 11 ms per 256 × 256 frame), enabling real-time image synthesis. Significance . The proposed framework provides an effective solution for real-time bidirectional MRI-US image translation under limited paired data. By combining physics-motivated data augmentation with an efficient GAN architecture, it achieves a favorable balance between image quality and computational efficiency, making it a promising approach for latency-sensitive applications such as ultrasound scanning simulation and intraoperative navigation.