OS-RRG: Observation State-Aware Radiology Report Generation With Balanced Diagnosis and Attention Intervention

Radiology report generation (RRG) aims to automatically generate detailed textual descriptions and diagnoses for clinical radiography, alleviating radiologists' workloads, aiding inexperienced radiologists, and minimizing errors. RRG is challenging due to the need to generate coherent and clinically accurate multisentence reports that describe various medical conditions. Although previous diagnosis-guided methods achieve impressive diagnostic accuracy by explicitly converting the identified observation states (OSs) (e.g., positive, negative, and uncertain) to descriptions, these methods still struggle in accurate observation-state identification and establishing precise state-to-description alignment. These challenges largely stem from the two aspects of imbalance (interclass and intraclass) inherent in observation states. In this article, we introduce a novel framework, observation state-aware radiology report generator (OS-RRG), designed to improve both the identification of states and their alignment with clinical descriptions. Our approach includes a state-aware balancing diagnosis (SBD) module to address both interclass and intraclass imbalances, an issue that previous methods have overlooked, resulting in suboptimal identification performance. In addition, we propose a novel technique called state-guided attention intervention (SAI), which dynamically adjusts focus on critical diagnostic features through a targeted filtering and enhancement mechanism. Furthermore, we propose a task-specific learning paradigm that decouples the identification and alignment processes into independent pathways, significantly enhancing the overall performance. Experiments on the MIMIC-CXR and IU-Xray benchmarks demonstrate the superior diagnostic accuracy of our method, which outperforms existing state-of-the-art techniques. The code will be made publicly available at https://github.com/xmed-lab/OS_RRG.