Ovarian–Adnexal Reporting and Data System (O-RADS) MRI Scoring: Diagnostic Accuracy, Interobserver Agreement, and Applicability to Machine Learning

Abstract Objectives To evaluate the interobserver agreement and diagnostic accuracy of ovarian-adnexal reporting and data system magnetic resonance imaging (O-RADS MRI) and applicability to machine learning. Methods Dynamic contrast-enhanced pelvic MRI examinations of 471 lesions were retrospectively analysed and assessed by 3 radiologists according to O-RADS MRI criteria. Radiomic data were extracted from T2 and post-contrast fat-suppressed T1-weighted images. Using these data, an artificial neural network (ANN), support vector machine, random forest, and naive Bayes models were constructed. Results Among all readers, the lowest agreement was found for the O-RADS 4 group (kappa: 0.669; 95% confidence interval [CI] 0.634-0.733), followed by the O-RADS 5 group (kappa: 0.709; 95% CI 0.678-0.754). O-RADS 4 predicted a malignancy with an area under the curve (AUC) value of 74.3% (95% CI 0.701-0.782), and O-RADS 5 with an AUC of 95.5% (95% CI 0.932-0.972) (P < .001). Among the machine learning models, ANN achieved the highest success, distinguishing O-RADS groups with an AUC of 0.948, a precision of 0.861, and a recall of 0.824. Conclusion The interobserver agreement and diagnostic sensitivity of the O-RADS MRI in assigning O-RADS 4-5 were not perfect, indicating a need for structural improvement. Integrating artificial intelligence into MRI protocols may enhance their performance. Advances in knowledge Machine learning can achieve high accuracy in the correct classification of O-RADS MRI. Malignancy prediction rates were 74% for O-RADS 4 and 95% for O-RADS 5.