An Automatic Recognition of Tooth- Marked Tongue Based on Tongue Region Detection and Tongue Landmark Detection via Deep Learning

The tooth-marked tongue refers to the tongue with the edge featured in jagged teeth marks, which is a significant indicator for reflecting the conditions of patients' internal organs in Traditional Chinese Medicine (TCM). From the perspective of computer vision, due to the small variance in the global region (original image) but the large variance in the local region (tongue region), especially in the differential region (tongue edge region around landmarks), the recognition of the tooth-marked tongue is a naturally fine-grained classification task. To address this challenging task, a two-stage method based on tongue region detection and tongue landmark detection via deep learning is proposed in this paper. In the first stage, we introduce a cascaded convolutional neural network to detect the tongue region and tongue landmarks simultaneously for minimizing the redundancy information and maximizing discriminative information explicitly. In the second stage, we send not only the detected tongue region but also the detected tongue landmarks to a fine-grained classification network for the final recognition. Conclusively, our method is highly consistent with human perception. Moreover, to the best of our knowledge, we are the first attempt to manage the tooth-marked tongue recognition via deep learning. We conducted extensive experiments with the proposed method. The experimental results demonstrate the effectiveness of the proposed method.