作者
Yishaoying Jia,Jinfu Niu,Zhonghui Qie,Zongyu Li,Andrew Laine,Jia Guo
摘要
Abstract Accurate classification of brain tumors from MRI is critical for guiding clinical decision-making; however, existing deep learning models are often hindered by limited interpretability and pronounced sensitivity to hyperparameter selection, which constrain their reliability in medical settings. To address these challenges, we propose TumorCLIP, a lightweight and training-efficient vision-language framework that integrates radiology-informed text prototypes with a DenseNet-based visual encoder to sup-port clinically meaningful semantic reasoning, fused via a Tip-Adapter mechanism. TumorCLIP does not aim to introduce a new vision-language model architecture. Instead, its contribution lies in the integration of radiology-informed text proto-types tailored to MRI interpretation, a systematic evaluation of backbone stability across diverse visual architectures, and a lightweight, training-efficient CLIP-based fusion framework designed for medical imaging applications. We first conduct a com-prehensive unimodal benchmark across eight representative visual backbones (EfficientNet-B0, MobileNetV3-Large, ResNet50, DenseNet121, ViT, DeiT, Swin Transformer, and MambaOut) using a standardized optimizer and learning-rate grid search, revealing performance swings exceeding 60 percentage points depending on hyperparameter choices. DenseNet121 shows the strongest stability-accuracy trade-off within our evaluated optimizer and learning-rate grid (97.6%). Leveraging this foundation, TumorCLIP fuses image features with frozen CLIP-derived text prototypes, achieving concept-level explainability, architectural support for zero-shot and few-shot inference, and enhanced classification of minority tumor classes. On the test set, Tu-morCLIP attains 98.5% accuracy and reduces inter-class misclassification compared to the unimodal DenseNet121 baseline. Semantic interpretability analysis further confirms that TumorCLIP achieves a positive semantic margin rate of 96.9% across all tumor categories, compared to 42.1% for the unimodal baseline, demonstrating that its embeddings are both discriminative and radiologically meaningful. Importantly, under cross-dataset evaluation, it demonstrates improved robustness with smaller performance degradation across tumor categories, particularly for clinically heterogeneous classes such as glioma. These results suggest that the framework maintains strong predictive performance while providing an explicit radiology-informed semantic reference page. TumorCLIP therefore provides a practical, interpretable, and data-efficient alternative to conventional visual classifiers. It highlights the role of radiology-aware vision-language alignment in MRI-based brain tumor classification. All results are reported within the evaluated datasets and training protocols.