Development and evaluation of machine learning models and nomogram for the prediction of severe acute pancreatitis

Abstract Background and Aim Severe acute pancreatitis (SAP) in patients progresses rapidly and can cause multiple organ failures associated with high mortality. We aimed to train a machine learning (ML) model and establish a nomogram that could identify SAP, early in the course of acute pancreatitis (AP). Methods In this retrospective study, 631 patients with AP were enrolled in the training cohort. For predicting SAP early, five supervised ML models were employed, such as random forest (RF), K ‐nearest neighbors (KNN), and naive Bayes (NB), which were evaluated by accuracy (ACC) and the areas under the receiver operating characteristic curve (AUC). The nomogram was established, and the predictive ability was assessed by the calibration curve and AUC. They were externally validated by an independent cohort of 109 patients with AP. Results In the training cohort, the AUC of RF, KNN, and NB models were 0.969, 0.954, and 0.951, respectively, while the AUC of the Bedside Index for Severity in Acute Pancreatitis (BISAP), Ranson and Glasgow scores were only 0.796, 0.847, and 0.837, respectively. In the validation cohort, the RF model also showed the highest AUC, which was 0.961. The AUC for the nomogram was 0.888 and 0.955 in the training and validation cohort, respectively. Conclusions Our findings suggested that the RF model exhibited the best predictive performance, and the nomogram provided a visual scoring model for clinical practice. Our models may serve as practical tools for facilitating personalized treatment options and improving clinical outcomes through pre‐treatment stratification of patients with AP.