A deep-learning-enabled diagnosis of ovarian cancer

We read with interest the results of the retrospective multicentre study on deep learning-enabled diagnosis of ovarian cancer by Yue Gao and colleagues,1Gao Y Zeng S Xu X et al.Deep learning-enabled pelvic ultrasound images for accurate diagnosis of ovarian cancer in China: a retrospective, multicentre, diagnostic study.Lancet Digit Health. 2022; 4: e179-e187Summary Full Text Full Text PDF Scopus (20) Google Scholar which is the largest study conducted on this topic to our knowledge. Nevertheless, we have several methodological concerns that warrant caution regarding the study's conclusions. First, although the goal of the deep convolutional neural network was to discriminate between benign and ovarian masses, the training dataset included healthy controls. These individuals do not belong to the target population (ie, patients with ovarian masses) and hence distort predictions for those who do.2Lijmer JG Mol BW Heisterkamp S et al.Empirical evidence of design-related bias in studies of diagnostic tests.JAMA. 1999; 282: 1061-1066Crossref PubMed Scopus (1496) Google Scholar Additionally, the number of healthy controls and their inclusion and exclusion criteria were not described. Second, the data curation process was not clearly described. Without adequate care, the model might leverage clinician-based information that is inherently present in retrospectively collected JPEG images, such as annotations, calipers, or Doppler information. If so, model performance might be overoptimistic and generalisability might be affected.3Badgeley MA Zech JR Oakden-Rayner L et al.Deep learning predicts hip fracture using confounding patient and healthcare variables.NPJ Digit Med. 2019; 2: 31Crossref PubMed Scopus (127) Google Scholar Interpretation methods, such as saliency maps, could give insight into this process and explain some of the black-box decisions that are made by the deep convolutional neural network. Third, the conclusion that the model outperforms the average diagnostic level of radiologists might be softened. The reported performance of the radiologists in the study is low compared with the performance that is described elsewhere in the scientific literature.4Meys EM Kaijser J Kruitwagen RF et al.Subjective assessment versus ultrasound models to diagnose ovarian cancer: a systematic review and meta-analysis.Eur J Cancer. 2016; 58: 17-29Summary Full Text Full Text PDF PubMed Scopus (157) Google Scholar Fourth, the calibration performance of the deep convolutional neural network was poor with underestimated risks of malignancy, which could lead to undertreatment when used in practice. Gao and colleagues1Gao Y Zeng S Xu X et al.Deep learning-enabled pelvic ultrasound images for accurate diagnosis of ovarian cancer in China: a retrospective, multicentre, diagnostic study.Lancet Digit Health. 2022; 4: e179-e187Summary Full Text Full Text PDF Scopus (20) Google Scholar refer to ultrasound-based risk prediction algorithms, such as the ADNEX logistic regression model, as "effective methods to discriminate between benign and malignant adnexal masses". Indeed, several external validation studies by investigators have reported AUCs above 0·9 with decent calibration performance (ie, with calibration curves close to the ideal diagonal line).5Van Calster B Valentin L Froyman W et al.Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study.BMJ. 2020; 370m2614Google Scholar In conclusion, we are concerned about the potential value and face validity of this model. We declare no competing interests. Deep learning-enabled pelvic ultrasound images for accurate diagnosis of ovarian cancer in China: a retrospective, multicentre, diagnostic studyThe performance of DCNN-enabled ultrasound exceeded the average diagnostic level of radiologists matched the level of expert ultrasound image readers, and augmented radiologists' accuracy. However, these observations warrant further investigations in prospective studies or randomised clinical trials. Full-Text PDF Open Access