Differentiating pancreatic ductal adenocarcinoma and autoimmune pancreatitis using a machine learning model based on ultrasound clinical features

Purpose This study aimed to construct a differential diagnostic model to distinguish autoimmune pancreatitis (AIP) from pancreatic ductal adenocarcinoma (PDCA) using ultrasound clinical features and machine learning algorithms. Methods Retrospective ultrasound clinical data of patients with AIP and PDCA from three different centers were used as the training cohort, external validation cohort 1, and external validation cohort 2. Feature selection was conducted via variance filtering and LASSO regression, followed by the construction of a random forest (RF) model. The hyperparameters were optimized in the training cohort, and the final model was evaluated in the external validation cohorts. The model’s performance was assessed using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), the F1 score, accuracy, and area under curve(AUC). The clinical application value of the model was clarified through a comparison between humans and machines. Results An RF model was constructed using six features: Ca 19-9 level, abdominal pain, jaundice, focal/diffuse-type AIP, blood flow signals, and morphology. In external validation cohort 1, the model’s sensitivity, specificity, PPV, NPV, F1 score, accuracy, and AUC were 86.0%, 80.0%, 81.0%, 86.0%, 84.0%, 83.0%, and 89.0%, respectively; in external validation cohort 2, these values were 72.0%, 94.0%, 93.0%, 77.0%, 81.0%, 83.0%, and 91.0%, respectively. The predictive performance of experienced radiologists using clinical information and ultrasound images demonstrated a sensitivity of 81%, specificity of 79%, PPV of 78%, NPV of 76%, F1 score of 80%, and accuracy of 80%. For radiologists with intermediate experience, the sensitivity was 75%, specificity was 74%, PPV was 73%, NPV was 76%, F1 score was 74%, and accuracy was 75%. less experienced radiologist had a sensitivity of 55%, specificity of 56%, PPV of 62%, NPV of 49%, F1 score of 58%, and accuracy of 55%. Conclusion The RF model constructed using clinical ultrasound features achieved diagnostic levels comparable to those of experienced radiologists and can assist in differentiating AIP from PDCA, potentially guiding clinical practice.