The performance of machine learning models in predicting suicidal ideation, attempts, and deaths: A meta-analysis and systematic review

Research has posited that machine learning could improve suicide risk prediction models, which have traditionally performed poorly. This systematic review and meta-analysis evaluated the performance of machine learning models in predicting longitudinal outcomes of suicide-related outcomes of ideation, attempt, and death and examines outcome, data, and model types as potential covariates of model performance. Studies were extracted from PubMed, Web of Science, Embase, and PsycINFO. A bivariate mixed effects meta-analysis and meta-regression analyses were performed for studies using machine learning to predict future events of suicidal ideation, attempts, and/or deaths. Risk of bias was assessed for each study using an adaptation of the Prediction model Risk Of Bias Assessment Tool. Narrative review included 56 studies, and analyses examined 54 models from 35 studies. The models achieved a very good pooled AUC of 0.86, sensitivity of 0.66 (95% CI [0.60, 0.72)], and specificity of 0.87 (95% CI [0.84, 0.90]). Pooled AUCs for ideation, attempt, and death were similar at 0.88, 0.87, and 0.84 respectively. Model performance was highly varied; however, meta-regressions did not provide evidence that performance varied by outcome, data, or model types. Findings suggest that machine learning has the potential to improve suicide risk detection, with pooled estimates of machine learning performance comparing favourably to performance of traditional suicide prediction models. However, more studies with lower risk of bias are necessary to improve the application of machine learning in suicidology.