CariesFG: A fine-grained RGB image classification framework with attention mechanism for dental caries

Dental caries is one of the most prevalent oral diseases, and deep learning methods have been used for caries diagnosis in large populations by leveraging RGB images. The existing attention-based fine-grained image classification methods have the problem of underutilization of features and easy interference by background and irrelevant information. To tackle these issues, we propose a fine-grained RGB image classification framework with attention mechanism for dental caries (CariesFG). Specifically, it consists of 4 components: (1) Multi-Spectral channel Attention Module (MSAM), which can retain the useful frequency components in the feature map. (2) Position Attention Module (PAM), which captures feature dependencies in the spatial dimension. (3) Discriminative Point Selection strategy (DPS), which can find the most discriminative feature points. (4) Graph Convolution and Aggregation module (GCA), which aims to aggregate discriminative feature points at different scales of feature maps. To enhance the ability to extract the discriminative features, PAM and MSAM are integrated into the backbone network to consist of feature extraction networks incorporating attention mechanism. Discriminative feature points at different scales of feature maps are extracted by DPS and aggregated as global discriminative features by GCA. By testing on a caries fine-grained classification dataset, CariesFG achieved an accuracy of 68.36%, an f1-score of 66.77% and a specificity of 84.17%, respectively, significantly outperforming state-of-the-art methods. Moreover, visualization results on attention parts show that CariesFG can effectively learn discriminative features and discriminative parts.