Increasing Adherence to Societal Recommendations in Radiology Reporting

Abstract Incidental findings in diagnostic imaging are common, but follow-up recommendations often lack consistency. The Society of Radiologists in Ultrasound (SRU) issued guidelines in 2021 for managing incidentally detected gallbladder polyps, aiming to balance follow-up with avoiding overtreatment. There is variable adherence to these guidelines in radiology reports, however, which makes it difficult for the clinician to pursue appropriate follow-up for the patient. The purpose of this project is to test the feasibility of a Large Language Model (LLM)–based tool to incorporate SRU guidelines into radiology reports. Additionally, we propose a framework for closely integrating societal follow-up recommendations into radiology reports, using this tool as an example. Following institutional review board approval, we retrospectively reviewed gallbladder ultrasound examinations performed on adult ED patients in 2022. Data on patient demographics and radiology report content were collected. Using the 2021 SRU guidelines, we developed an interactive tool employing a retriever-augmented generator (RAG) and prompt engineering. A board-certified radiologist tested the accuracy, whereas a board-certified emergency medicine physician assessed the clarity and consistency of the recommendations. The interactive tool, GB-PRL, outperformed leading closed-source and open-source LLMs, achieving 100% accuracy in risk categorization and follow-up recommendations on hypothetical user queries ( P < 0.001). The tool also showed superior accuracy compared to radiology reports on retrospective data ( P = 0.04). Although GB-PRL demonstrated greater clarity and consistency, the improvement from the radiology reports was not statistically significant ( P = 0.22). Further work is needed for prospective testing of GB-PRL before integrating it into clinical practice.