Intelligent tongue diagnosis model for gastrointestinal diseases based on tongue images

Currently, the diagnosis of gastrointestinal diseases relies heavily on electronic endoscopy, which is not suitable for all individuals and insufficient for dynamic patient monitoring. Therefore, there is an urgent need for a convenient and noninvasive examination method to enhance the diagnosis and monitoring of gastrointestinal diseases. Since the tongue is a significant reflection of the digestive system and closely associated with gastrointestinal diseases, our study aims to develop a diagnostic framework based on tongue features. We have collected a dataset of 2,167 tongue coating images from 949 consecutive patients, including 905 images from healthy individuals and 1,262 images from patients with various gastrointestinal conditions. Notably, each patient sample may represent distinct gastrointestinal diseases. This study introduces a novel approach to information fusion detection, integrating hand-crafted and auto-encoded features, alongside disease detection employing Squeeze and Excitation (SE) and Slot attention mechanisms. These two branches respectively analyze image feature and spatial position information when utilizing tongue images for diagnosing gastrointestinal diseases. The amalgamation of predictions from both aspects yields the final detection outcome. For both healthy and diseased cohorts, the optimal classification metrics were achieved: AUC = 0.886, ACC = 0.849, and TPR = 0.965, indicating satisfactory performance. Notably, our framework demonstrated promising results in detecting five common gastrointestinal diseases: Helicobacter pylori infection, bile reflux, reflux esophagitis, gastric erosion, and duodenal erosion. Ablation experiments underscored the efficacy of mixed features in multi-color space, showcasing an increase in AUC and ACC by 13.48 % and 12.05 % respectively. Furthermore, the attention mechanism, by focusing on the middle region of tongue images, contributed to a 6.12 % increase in AUC and a 6.35 % increase in ACC. In this study, we successfully develop and validate a diagnosis framework for gastrointestinal diseases based on tongue image features. By utilizing tongue images, we achieve intelligent diagnosis of gastrointestinal diseases, establishing tongue diagnosis as a convenient, cost-effective, and noninvasive method for diagnosing and screening gastrointestinal diseases.