Application of Quantitative Interpretability to Evaluate CNN-Based Models for Medical Image Classification

Convolutional Neural Networks (CNNs) dominate medical image classification, yet their “black box” nature limits understanding of their decision-making process. This study applies quantitative interpretability metrics to evaluate CNN performance in stained tongue coating recognition and compare with traditional metrics. We trained four classical CNN models (ResNet18, ResNet50, VGG19, and AlexNet) on a dataset of 2,008 tongue coating images, with external validation on 381 new images. Class Activation Mapping (CAM) algorithms generated heatmaps visualizing influential regions. The Heatmap Assisted Accuracy Score (HAAS) was utilized to assess feature attribution quality. All models achieved high classification performance on the test set (accuracy >0.92, precision >0.89, recall >0.91), but VGG19 and AlexNet performed poorly on external validation. Interpretability analysis revealed that VGG19 and AlexNet deviated from regions of interest, while ResNet models achieved significantly higher HAAS scores. ResNet50 emerged as the best model in external validation (accuracy =0.900, precision =0.869, recall =0.911), consistent with its superior interpretability metrics (Eigen-CAM HAAS=1.548). Our findings demonstrate that interpretability metrics more accurately reflect CNN performance before external validation, offering valuable tools for understanding model behavior and enhancing transparency in medical image classification.