Intelligent difficulty scoring and assistance system for endoscopic extraction of common bile duct stones based on deep learning: multicenter study

Abstract Background The study aimed to construct an intelligent difficulty scoring and assistance system (DSAS) for endoscopic retrograde cholangiopancreatography (ERCP) treatment of common bile duct (CBD) stones. Methods 1954 cholangiograms were collected from three hospitals for training and testing the DSAS. The D-LinkNet34 and U-Net were adopted to segment the CBD, stones, and duodenoscope. Based on the segmentation results, the stone size, distal CBD diameter, distal CBD arm, and distal CBD angulation were estimated. The performance of segmentation and estimation was assessed by mean intersection over union (mIoU) and average relative error. A technical difficulty scoring scale, which was used for assessing the technical difficulty of CBD stone removal, was developed and validated. We also analyzed the relationship between scores evaluated by the DSAS and clinical indicators including stone clearance rate and need for endoscopic papillary large-balloon dilation (EPLBD) and lithotripsy. Results The mIoU values of the stone, CBD, and duodenoscope segmentation were 68.35 %, 86.42 %, and 95.85 %, respectively. The estimation performance of the DSAS was superior to nonexpert endoscopists. In addition, the technical difficulty scoring performance of the DSAS was more consistent with expert endoscopists than two nonexpert endoscopists. A DSAS assessment score ≥ 2 was correlated with lower stone clearance rates and more frequent EPLBD. Conclusions An intelligent DSAS based on deep learning was developed. The DSAS could assist endoscopists by automatically scoring the technical difficulty of CBD stone extraction, and guiding the choice of therapeutic approach and appropriate accessories during ERCP.