Assessment of performance of the machine learning-based breast cancer risk prediction models: a systematic review.

Background: Several studies have explored the predictive performance of machine learning-based breast cancer risk prediction models and the results arrived at controversial conclusions, which prompts us to review the performance and weaknesses of machine learning-based breast cancer risk prediction models.Objectives: To assess the performance and clinical feasibility of available machine learning-based breast cancer risk prediction model.Methods: As of June 9, 2021, articles on breast cancer risk prediction models by machine learning were searched in PubMed, Embase, and Web of Science. Studies describing the development or validation models for predicting future breast cancer risk were included. Prediction Model Risk of Bias Assessment Tool (PROBAST) was used to assess the risk of bias (ROB) and clinical applicability of included studies. Pooled area under the curve (AUC) were calculated using the DerSimonian and Laird random-effects model.Results: A total of 8 studies with 10 datasets were included. Neural network was the most common machine learning method for the development of breast cancer risk prediction models. The pooled AUC of machine learning-based optimal risk prediction model reported in each study was 0.73 (95%CI: 0.66-0.80; approximate 95%PI: 0.56-0.96), with a high level of heterogeneity between studies (Q value=576.07, I2=98.44%, P < .001). The results of head-to-head comparison the performance difference in two type models trained by same dataset showed that machine learning models had a slight advantage in predicting future breast cancer risk than traditional risk factor-based models. The pooled AUC of neural network-based risk prediction model was higher than that of non-neural network-based optimal risk prediction model (0.71 vs. 0.68). Subgroup analysis showed that incorporation of imaging features risk models had a higher pooled AUC than model of non-incorporation of imaging features (0.73 vs. 0.61; Pheterogeneity = .001). The PROBAST analysis indicated that many machine learning models had high ROB, and poorly reported calibration analysis.Conclusions: Machine learning-based breast cancer risk prediction models had some technical pitfalls, and their clinical feasibility and reliability were unsatisfactory.