Constructing a Unified Vision-Language Model for Chest Radiograph–based Diagnostics, Medical Education, and Data Augmentation

Purpose To develop MedXChat, a large language model (LLM) capable of integrating radiology report generation, visual question answering (VQA), and text-to-image synthesis and evaluate its performance via computational metrics and expert radiologist assessments. Materials and Methods In this retrospective study, MedXChat was trained on the MIMIC Chest X-ray (MIMIC-CXR) database, comprising 270 790 chest radiograph-report pairs, 54 138 VQA samples, and 7500 text-to-image instruction pairs. Data were collected from 2011 to 2016. Computational evaluations of MedXChat performance were conducted using the F1 score, area under the receiver operating characteristic curve (AUC), and Fréchet inception distance (FID). Radiologist evaluations involved six experts-three junior, two senior, and one supervisor-who assessed 50 random MedXChat outputs for accuracy, consistency, and alignment with clinical standards. Results In the chest radiograph-to-report test set, MedXChat achieved an AUC of 0.67 (95% CI: 0.61, 0.75), higher than UniXGen (AUC, 0.54; P < .001) and LLM-CXR (AUC, 0.63; P = .02). Its F1 score was 0.44 versus 0.26 (P < .001) and 0.41 (P = .04), respectively. In chest radiograph-VQA, MedXChat showed higher accuracy for edema (73% vs 54% for LLM-CXR and 60% for LLaVA-Med) and pleural effusion (80% vs 53% for LLM-CXR and 61% for LLaVA-Med; all P ≤ .01). In text-to-image synthesis, it achieved the lowest FID (43.46 vs 73.29 and 106.17; P < .001) and the highest classification accuracy (71.5% vs 68.6% and 67.2%; P ≤ .05), producing high-quality images including lateral views. Conclusion MedXChat integrated report generation, VQA, and image synthesis within a unified framework, achieving state-of-the-art performance. MedXChat may support future professional applications and enhance radiologic workflows, education, and data augmentation. Keywords: Computer Aided Diagnosis (CAD), Applications - Decision Support, Applications - Multimodal, Outcomes Analysis, Technology Assessment, Comparative Studies Supplemental material is available for this article. © RSNA, 2025.