Supervised and semi-supervised training of deep convolutional neural networks for gastric landmark detection

This work focuses on detection of upper gas-trointestinal (GI) landmarks, which are important anatomical areas of the upper GI tract digestive system that should be photodocumented during endoscopy to guarantee a complete examination. The aim of this work consisted in testing new automatic algorithms, specifically based on convolutional neural network (CNN) systems, able to detect upper GI landmarks, that can help to avoid the presence of blind spots during esophagogastroduodenoscopy. We tested pre-trained CNN architectures, such as the ResNet-50 and VGG-16, in conjunction with different training approaches, including the use of class weights, batch normalization, dropout, and data augmentation. The ResNet-50 model trained with class weights was the best performing CNN, achieving an accuracy of 71.79% and a Mathews Correlation Coefficient (MCC) of 65.06%. The combination of supervised and unsupervised learning was also explored to increase classification performance. In particular, convolutional autoencoder architectures trained with unlabeled GI images were used to extract representative features. Such features were then concatenated with those extracted by the pre-trained ResNet-50 architecture. This approach achieved a classification accuracy of 72.45% and an MCC of 65.08%. Clinical relevance— Esophagogastroduodenoscopy (EGD) photodocumentation is essential to guarantee that all areas of the upper GI system are examined avoiding blind spots. This work has the objective to help the EGD photodocumentation monitorization by testing new CNN-based systems able to detect EGD landmarks