Diagnostic Performance of O‐RADS US (Version 2019 and Version 2022) Incorporating Acoustic Shadowing by Junior Radiologists

Introduction Acoustic shadowing is an important benign ultrasound (US) feature for adnexal masses (AMs). To validate the diagnostic performance and interobserver agreement of the 2019 version and 2022 version of Ovarian‐Adnexal Reporting and Data System Ultrasound (O‐RADS US) and ascertain whether adding acoustic shadowing to O‐RADS US v2019 as a benign ultrasound feature can enhance its diagnostic efficacy among junior radiologist. Methods This retrospective study included consecutive women with suspected adnexal masses who underwent ultrasound examinations between September 2022 and January 2024. One junior doctor (Reader 1, 2 years of experience) classified each AM according to the O‐RADS US v2019 and the v2022. Lesions were reclassified according to the US features of acoustic shadowing based on the O‐RADS US v2019: the O‐RADS category was downgraded by one level with acoustic shadowing and remained unchanged without acoustic shadowing for categories 2–5. Receiver operating characteristic (ROC) curve analysis was used to evaluate the diagnostic performance of the two versions of O‐RADS and the modified O‐RADS model. Two independent junior radiologists (Reader 1 and Reader 2 with 3 years of experience) then classified the 200 AMs randomly selected for a test–retest analysis. Kappa ( к ) statistics were used to assess the interobserver agreement. Results Overall, 1015 women (range, 16–86 years) with 1061 AMs were evaluated. Of the lesions, 864 (81.4%) were benign and 197 (18.6%) were malignant. The area under the ROC curve (AUC) for O‐RADS v2019 and v2022 were 0.920 (95% confidence interval [CI]: 0.902, 0.936, P < .001) and 0.924 (95% CI: 0.906, 0.939, P < .001), respectively. The modified model based on O‐RADS v2019 incorporating acoustic shadowing as a benign US feature showed an improved AUC of 0.934 (95% CI: 0.917, 0.948, P < .001). A significant difference was observed between the AUCs of the modified O‐RADS and two versions of O‐RADS models ( P < .005). The inclusion of acoustic shadowing increased specificity by 5.4% in predicting malignant adnexal masses, compared with the O‐RADS US v2019 with a specificity of 76.2%. Using the modified O‐RADS category 4 as the optimal cut‐off value for predicting malignancy showed a sensitivity, specificity, positive predictive value, and negative predictive value were 94.4% (95% CI: 90.2%, 97.2%), 81.6% (95% CI: 78.8%, 84.1%), 53.9% (95% CI: 50.3%, 57.5%), and 98.5% (95% CI: 97.3%, 99.1%), respectively. The inter‐observer agreement in the O‐RADS category between these two junior radiologists was good ( κ = 0.74, P < .001). Conclusion We validated the excellent performance of the Ovarian‐Adnexal Reporting and Data system Ultrasound for diagnosing adnexal masses, and the inclusion of acoustic shadowing increased specificity by 5.4% in predicting malignant adnexal masses, compared with the O‐RADS US v2019 with the specificity of 76.2%.