Diagnostic performance and reproducibility of ACR Bone-RADS for the radiographic evaluation of potentially neoplastic bone lesions

To assess the diagnostic performance and the reproducibility of the American College of Radiology (ACR) Bone-RADS score for the radiographic evaluation of bone lesions. Patients with histologically proven bone lesion of the extremities and radiographs performed within three months prior to biopsy/surgery were retrospectively enrolled. All radiographs were evaluated using ACR Bone-RADS scoring system by two musculoskeletal radiologists (Readers 1 and 2, with 10 and 3 years of experience in bone tumors imaging, respectively), and one junior general radiologist (Reader 3). Percent agreement, Cohen's Kappa and Conger's Kappa were calculated for all score categories. Weighted-kappa and intraclass correlation coefficient (ICC) were computed for the final Bone-RADS score. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated based on the expert radiologist's diagnosis, using histology as the reference standard. Final cohort consisted of 285 patients. Substantial to almost perfect agreement was found on almost all Bone-RADS categories. Moderate agreement was found for "endosteal scalloping" between Readers 1-3 (K = 0.59) and Readers 2-3 (K = 0.55). The score showed good diagnostic performance in discriminating between benign and potentially aggressive lesions (AUC = 0.82). Suboptimal sensitivity and NPV were found using a Bone-RADS ≥ 3 cut-off; if enchondroma and atypical cartilaginous tumors were excluded, diagnostic performance significantly improved (AUC = 0.92). The ACR Bone-RADS score showed high reproducibility and good accuracy in identifying benign lesions. In cases of cartilaginous tumors, the score may underestimate the disease, suggesting that additional imaging, rather than radiographic follow-up, could be considered for Bone-RADS-2 chondroid lesions.