Artificial Intelligence Performance in Image-Based Cancer Identification: Umbrella Review of Systematic Reviews

Background Artificial intelligence (AI) has the potential to transform cancer diagnosis, ultimately leading to better patient outcomes. Objective We performed an umbrella review to summarize and critically evaluate the evidence for the AI-based imaging diagnosis of cancers. Methods PubMed, Embase, Web of Science, Cochrane, and IEEE databases were searched for relevant systematic reviews from inception to June 19, 2024. Two independent investigators abstracted data and assessed the quality of evidence, using the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Systematic Reviews and Research Syntheses. We further assessed the quality of evidence in each meta-analysis by applying the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) criteria. Diagnostic performance data were synthesized narratively. Results In a comprehensive analysis of 158 included studies evaluating the performance of AI algorithms in noninvasive imaging diagnosis across 8 major human system cancers, the accuracy of the classifiers for central nervous system cancers varied widely (ranging from 48% to 100%). Similarities were observed in the diagnostic performance for cancers of the head and neck, respiratory system, digestive system, urinary system, female-related systems, skin, and other sites. Most meta-analyses demonstrated positive summary performance. For instance, 9 reviews meta-analyzed sensitivity and specificity for esophageal cancer, showing ranges of 90%-95% and 80%-93.8%, respectively. In the case of breast cancer detection, 8 reviews calculated the pooled sensitivity and specificity within the ranges of 75.4%-92% and 83%-90.6%, respectively. Four meta-analyses reported the ranges of sensitivity and specificity in ovarian cancer, and both were 75%-94%. Notably, in lung cancer, the pooled specificity was relatively low, primarily distributed between 65% and 80%. Furthermore, 80.4% (127/158) of the included studies were of high quality according to the JBI Critical Appraisal Checklist, with the remaining studies classified as medium quality. The GRADE assessment indicated that the overall quality of the evidence was moderate to low. Conclusions Although AI shows great potential for achieving accelerated, accurate, and more objective diagnoses of multiple cancers, there are still hurdles to overcome before its implementation in clinical settings. The present findings highlight that a concerted effort from the research community, clinicians, and policymakers is required to overcome existing hurdles and translate this potential into improved patient outcomes and health care delivery. Trial Registration PROSPERO CRD42022364278; https://www.crd.york.ac.uk/PROSPERO/view/CRD42022364278