UC-NfNet: Deep learning-enabled assessment of ulcerative colitis from colonoscopy images

Ulcerative colitis (UC) belongs to the inflammatory bowel disease (IBD) family, which is mainly caused by inflammation of the tissue in the colon and rectum. The severity of this infection can radically affect the patient's overall well-being. Although there is no definitive treatment for this disease, diagnosis of the severity of the disease through colonoscopy imaging and the use of personalized treatment can prevent progression to more malignant stages. Inter- and intra-observer variability combined with the complex nature of UC infection makes medical assessment cumbersome. Diagnosis and treatment of UC can be made more accurate and robust if disease severity can be determined in a standardized and automated manner. Therefore, the development of a computerized tool that can be integrated into the clinical decision-making process of UC classification is of great importance. In this work, we present an automated UC classification method, UC-NfNet, complemented by a synthetic data generation pipeline aimed at classifying colonoscopy UC images. We show that our model quantitatively outperforms state-of-the-art classification models such as ConViT, Inception-v4, NFNets, ResNets and Swin Transformer. In an independent reader study of five gastroenterologists, the average agreement between the UC-NfNet and individual gastroenterologists was higher than the agreement between individual gastroenterologists. This robust evaluation of the proposed AI system paves the way for clinical trials of AI-assisted UC classification. The code and dataset are publicly available at https://github.com/DeepMIALab/UC-NfNet.