Multimodal Models Are Still a Novice at Radiology Vision

HomeRadiologyVol. 312, No. 3 PreviousNext Reviews and CommentaryEditorialMultimodal Models Are Still a Novice at Radiology VisionFrancis Deng Francis Deng Author AffiliationsFrom the Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, 600 N Wolfe St, Baltimore, MD 21287.Address correspondence to the author (email: [email protected]).Francis Deng Published Online:Sep 3 2024https://doi.org/10.1148/radiol.242286See also the article by Hayden et al in this issue.MoreSectionsFull textPDF ToolsAdd to favoritesCiteTrack CitationsPermissionsReprints ShareShare onFacebookXLinked In References1. Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health 2023;2(2):e0000198. MedlineGoogle Scholar2. Bhayana R, Krishna S, Bleakney RR. Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations. Radiology 2023;307(5):e230582. Google Scholar3. Bhayana R, Bleakney RR, Krishna S. GPT-4 in Radiology: Improvements in Advanced Reasoning. Radiology 2023;307(5):e230987. Google Scholar4. Hayden N, Gilbert S, Poisson LM, Griffith B, Klochko C. Performance of GPT-4 with Vision on Text- and Image-based ACR Diagnostic Radiology In-Training Examination Questions. Radiology 2024;312(3):e240153. Google Scholar5. Wu C, Lei J, Zheng Q, et al. Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosis. arXiv 2310.09909 [preprint] https://arxiv.org/abs/2310.09909. Published October 15, 2023. Accessed July 30, 2024. Google Scholar6. Patel MD, Hunt KN, Benefield T, et al. The Relationship Between ACR Diagnostic Radiology In-Training Examination Scores and ABR Core Examination Outcome and Performance: A Multi-Institutional Study. J Am Coll Radiol 2020;17(12):1663–1669. MedlineGoogle Scholar7. Payne DL, Purohit K, Borrero WM, et al. Performance of GPT-4 on the American College of Radiology In-training Examination: Evaluating Accuracy, Model Drift, and Fine-tuning. Acad Radiol 2024;31(7):3046–3054. MedlineGoogle Scholar8. Zhou Y, Ong H, Kennedy P, et al. Evaluating GPT-4V (GPT-4 with Vision) on Detection of Radiologic Findings on Chest Radiographs. Radiology 2024;311(2):e233270. Google Scholar9. Suh PS, Shim WH, Suh CH, et al. Comparing Diagnostic Accuracy of Radiologists versus GPT-4V and Gemini Pro Vision Using Image Inputs from Diagnosis Please Cases. Radiology 2024;312(1):e240273. Google Scholar10. Hello GPT-4o.https://openai.com/index/hello-gpt-4o/. Published May 13, 2024. Accessed July 29, 2024. Google ScholarArticle HistoryReceived: July 31 2024Revision requested: Aug 9 2024Revision received: Aug 9 2024Accepted: Aug 9 2024Published online: Sept 03 2024 FiguresReferencesRelatedDetailsAccompanying This ArticlePerformance of GPT-4 with Vision on Text- and Image-based ACR Diagnostic Radiology In-Training Examination QuestionsSep 3 2024RadiologyRecommended Articles Wearable Magnetoencephalography: Reality or Science Fiction?Radiology2022Volume: 304Issue: 2pp. 435-436Deep Learning Algorithms to Detect Fractures: Systematic Review Shows Promising Results but Many LimitationsRadiology2022Volume: 304Issue: 1pp. 63-64Improved Vertebral Fracture Assessment: The Game-Changing Potential of Deep Learning with Multidetector CTRadiology2024Volume: 310Issue: 3Psychoradiology: The Frontier of Neuroimaging in PsychiatryRadiology2016Volume: 281Issue: 2pp. 357-372Diagnosis of Coronavirus Disease 2019 Pneumonia by Using Chest Radiography: Value of Artificial IntelligenceRadiology2020Volume: 298Issue: 2pp. E88-E97See More RSNA Education Exhibits Imaging in Non-accidental Trauma - A Primer for TraineesDigital Posters2022Deep Brain Stimulation (DBS) - Pre and Post Operative Imaging, From Adequate Nucleus and Fiber Tract Localization to Electrode MisplacementDigital Posters2022Cracking the Enigma: A Systematic Imaging Approach to Fracture Non-UnionDigital Posters2020 RSNA Case Collection COVID-19 PneumoniaRSNA Case Collection2020Monteggia Fracture Bado IIIRSNA Case Collection2021Subcapital Femoral Neck FractureRSNA Case Collection2021 Vol. 312, No. 3 Metrics Altmetric Score PDF download