Development of Convolutional Neural Network Models That Recognize Normal Anatomical Structures During Real-Time Radial- and Linear-Array Endoscopic Ultrasound (with Videos)

Endoscopic ultrasound (EUS) is a high-skill technique that requires numerous procedures to achieve competence. However, there are limited number of training facilities worldwide. Convolutional neural network (CNN) models have been previously implemented for object detection. We aimed to develop two EUS-based CNN models for normal anatomical structure recognition during real-time linear- and radial-array EUS evaluations.The study was performed from February 2020 to June 2022. Consecutive patient videos of linear- and radial-array EUS videos were recorded. Expert endosonographers identified and labeled twenty normal anatomical structures within the videos for training and validation of the CNN models. Initial CNN models (CNNv1) were developed from forty-five videos, and the improved models (CNNv2) from an additional 102 videos. The performance of the CNN models was compared to that of two expert endosonographers.CNNv1 used 45034 linear-array EUS frames and 21063 radial-array EUS frames. CNNv2 used 148980 linear-array EUS frames and 128871 radial-array EUS frames. CNNv1-L and CNNv1-R achieved a 75.65% and 71.36% mean average precision (mAP) with a total loss of 0.19 and 0.18, respectively. CNNv2-L obtained an 88.7% mAP with a 0.06 total loss, while CNNv2-R achieved an 83.5% mAP with a 0.07 total loss. The CNNv2 accurately detected all studied normal anatomical structures with >98% observed agreement during clinical validation.The proposed CNN models accurately recognize the normal anatomical structures in prerecorded videos and real-time EUS. Prospective trials are needed to evaluate the impact of these models on the learning curves of EUS trainees.