Development of an AI‐Assisted System for Automatic Recognition and Localization Marking of Colonic Polyps (With Video)

ABSTRACT Background Localizing colorectal polyps identified during the initial colonoscopy in minimally invasive endoscopic surgery presents significant challenges. These challenges include imprecise location descriptions, unclear images, a high number of polyps, and polyp characteristics such as flat shapes and low color contrast. To address these issues, we developed an AI‐assisted system for the automatic detection and localization of colorectal polyps. Methods Colonic images and videos from three medical centers, collected between January 2018 and August 2024, were categorized based on pathology results into normal, adenomatous polyp, and serrated lesion groups. Transfer learning and fine‐tuning were conducted on five pretrained CNN models, with performance evaluated using metrics such as accuracy, precision, sensitivity, and AUC. The best‐performing model was selected for interpretability analysis and developed into an AI‐assisted system capable of both polyp recognition and location marking. Results Among the five models, EfficientNetV2 performed the best, achieving accuracy, precision, sensitivity, and F1 scores of 0.933, 0.917, 0.916, and 0.917, respectively, on the validation set. On the test set, the model's overall weighted average precision, specificity, and AUC were 0.903, 0.946, and 0.983, respectively. Two representative colonoscopy case videos predicted by the model further demonstrated the feasibility of this AI system in clinical practice. Conclusions The AI system we developed for the automatic recognition and localization marking of colonic polyps in colonoscopy aids in the rapid localization of polyps during minimally invasive endoscopic surgery.