Interreader Agreement of Lung-RADS: A Systematic Review and Meta-Analysis

Background: Lung-RADS has shown variable interreader agreement in the literature, in part related to a broad range of factors that may influence the consistency of its implementation. Objective: To assess the interreader agreement of Lung-RADS and to investigate factors influencing the system's variability. Evidence Acquisition: EMBASE, PubMed, and Cochrane databases were searched for original research studies published through June 18, 2024 reporting the interreader agreement of Lung-RADS on chest CT. Random-effect models were used to calculate pooled kappa coefficients for Lung-RADS categorization and pooled intraclass correlation coefficients (ICCs) for nodule size measurements. Potential sources of heterogeneity were explored using metaregression analyses. Evidence Synthesis: The analysis included 11 studies (1470 patients) for Lung-RADS categorization and five studies (617 patients) for nodule size measurement. Interreader agreement for Lung-RADS categorization was substantial (κ=0.72 [95% CI, 0.57-0.82]), and for nodule size measurement was almost perfect (ICC=0.97 [95% CI, 0.90-0.99]). Interreader agreement for Lung-RADS categorization was significantly associated with the method of nodule measurement (p=.005), with pooled kappa coefficients for studies using computer-aided detection (CAD)-based semiautomated volume measurements, using CAD-based semiautomated diameter measurements, and using manual diameter measurements of 0.95, 0.91, and 0.66, respectively. Interreader agreement for Lung-RADS categorization was also significantly associated with studies' nodule type distribution (p<.001), with pooled kappa coefficients for studies evaluating 100% solid nodules, 30-99% solid nodules, and <30% solid nodules of 0.85, 0.76, and 0.55, respectively. Interreader agreement for nodule size measurement was significantly associated with radiation dose (p<.001), with pooled ICCs for studies that used standard-dose CT, used low-dose CT, and used ultralow-dose CT of 0.97, 0.96, and 0.59, respectively. Interreader agreement for nodule size measurement was also significantly associated with the Lung-RADS version used (p=.02), with pooled ICCs for studies using Lung-RADS 1.1 and using Lung-RADS 1.0 of 0.99 and 0.93, respectively. Conclusion: While supporting the overall reliability of Lung-RADS, the findings indicate roles for CAD assistance as well as training and standardized approaches for nodule type characterization to further promote reproducible application. Clinical Impact: Consistent nodule assessments will be critical for Lung-RADS to optimally impact patient management and outcomes.