Multiple abnormality classification in wireless capsule endoscopy images based on EfficientNet using attention mechanism

The wireless capsule endoscopy (WCE) procedure produces tens of thousands of images of the digestive tract, for which the use of the manual reading process is full of challenges. Convolutional neural networks are used to automatically detect lesions in WCE images. However, studies on clinical multilesion detection are scarce, and it is difficult to effectively balance the sensitivity to multiple lesions. A strategy for detecting multiple lesions is proposed, wherein common vascular and inflammatory lesions can be automatically and quickly detected on capsule endoscopic images. Based on weakly supervised learning, EfficientNet is fine-tuned to extract the endoscopic image features. Combining spatial features and channel features, the proposed attention network is then used as a classifier to obtain three classifications. The accuracy and speed of the model were compared with those of the ResNet121 and InceptionNetV4 models. It was tested on a public WCE image dataset obtained from 4143 subjects. On the computer-assisted diagnosis for capsule endoscopy database, the method gives a sensitivity of 96.67% for vascular lesions and 93.33% for inflammatory lesions. The precision for vascular lesions was 92.80%, and that for inflammatory lesions was 95.73%. The accuracy was 96.11%, which is 1.11% higher than that of the latest InceptionNetV4 network. Prediction for an image only requires 14 ms, which balances the accuracy and speed comparatively better. This strategy can be used as an auxiliary diagnostic method for specialists for the rapid reading of clinical capsule endoscopes.