P0500 Multi-Module Metaformer System for Distinguishing Crohn’s Disease from Intestinal Tuberculosis

Abstract Background Distinguishing Crohn’s Disease (CD) from Intestinal Tuberculosis (ITB) through endoscopy can be particularly challenging, especially for junior physicians or those in resource-limited settings. Accurate differentiation is essential for appropriate treatment and effective patient management. This study aims to develop a multimodal AI-assisted diagnostic model that integrates clinical data and colonoscopy images to reliably distinguish between CD and ITB. Methods Patients diagnosed with CD or ITB at Xiangya Third Hospital of Central South University and Hunan Chest Hospital between January 1, 2005, and August 1, 2023, were included in the study. An external validation set was obtained from Zhuzhou Central Hospital. Data collection involved clinical text information and colonoscopy images for each patient. A multimodal diagnostic model, based on the MetaFormer algorithm, was developed for differential diagnosis, which was evaluated using both internal and external validation sets. Model performance was assessed using accuracy, sensitivity, specificity, and F1 score, and compared to the performance of endoscopists. Results The analysis included 408 patients in the internal dataset (311 with CD and 97 with ITB) and 85 patients in the external validation set (62 with CD and 23 with ITB). The multimodal algorithm outperformed approaches that used a single data type. Compared to state-of-the-art CNN-based algorithms such as VILBERT, BLIP, and CLIP, the multimodal approach demonstrated significant superiority in performance metrics. In the internal dataset, the integrated model achieved 91.25% accuracy, 96.72% sensitivity, and 73.68% specificity, thereby effectively reducing misdiagnosis rates. The robustness of the model was confirmed through external validation, with results showing 91.76% accuracy, 95.16% sensitivity, and 82.61% specificity. The F1 score was calculated as 0.944. The model performance significantly outperforms different levels of endoscopist performance. Junior endoscopists had lower diagnostic accuracy compared to experts (76.24% vs. 85.15%), but using the model to assist junior endoscopists improved their diagnostic accuracy to levels comparable with expert endoscopists (83.17% vs. 85.15%). This indicates the model’s potential to enhance diagnostic accuracy and consistency in clinical practice. Conclusion The MetaFormer-based multimodal predictive model for differentiating CD and ITB demonstrates high discriminatory power, providing an accurate diagnostic tool to support clinical decision-making and improve patient outcomes. References [1]Vaswani A , Shazeer N , Parmar N ,et al.Attention Is All You Need[J].arXiv, 2017; DOI:10.48550/arXiv.1706.03762. [2]Wagner, S.J., et al., Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study. Cancer Cell, 2023; 41(9): p. 1650-1661 e4. [3]He, Y., et al., Development and Validation of a Novel Diagnostic Nomogram to Differentiate Between Intestinal Tuberculosis and Crohn’s Disease: A 6-year Prospective Multicenter Study. Am J Gastroenterol, 2019; 114(3): p. 490-499.