人工智能
计算机科学
脑瘤
模式识别(心理学)
计算机视觉
磁共振成像
医学影像学
特征提取
神经影像学
实时核磁共振成像
神经生理学
大脑定位
人工神经网络
计算机断层摄影术
图像处理
深度学习
作者
Zhibing Qiu,Yuxiao Liu,Jianrui Li,Lei Jin,Zhongxiang Ding,Feng Shi,Dinggang G. Shen
标识
DOI:10.1109/tmi.2026.3718445
摘要
Accurate MRI-based brain tumor analysis requires not only tumor subtype classification but also localization at an anatomical granularity that is consistent with radiology reports. Most vision-only methods address localization and classification as separate label-prediction tasks, and therefore provide limited alignment with the fine-grained anatomical semantics used in routine reporting. To address this limitation, we propose LIGHT (Learning Image-text Grounding for Hierarchical Tumor analysis), a 3D vision-language framework that formulates brain tumor localization and subtype classification as image-text retrieval in a shared embedding space. Given multimodal MRI inputs, LIGHT retrieves coarse anatomical regions, fine-grained subregions, and tumor subtypes in a unified coarse-to-fine procedure. The framework has three main components: 1) a hierarchical retrieval vocabulary containing 21 coarse-grained regions, 563 fine-grained subregions, and 5 tumor subtypes; 2) large-scale foundation pretraining on a curated patient-disjoint in-house dataset of 99,813 MRI-report pairs; and 3) grounded task fine-tuning that uses segmentation-guided tumor crops and multi-template prompt supervision to improve tumor-prompt alignment. Across 11,034 annotated 3D brain tumor MRI cases from in-house, external clinical, and public datasets, LIGHT achieved 72.1% accuracy for coarse-grained localization, 70.1% top-1 accuracy for fine-grained localization, and 76.0%, 83.7%, and 84.6% accuracy for subtype classification on three 3D datasets, respectively. Additional public 2D subtype-classification experiments provide further evidence of cross-setting subtype recognition. These results suggest that report-grounded image-text retrieval can provide interpretable anatomical and diagnostic outputs for brain tumor MRI analysis. Code: https://github.com/qiuzhaoyu/LIGHT.
科研通智能强力驱动
Strongly Powered by AbleSci AI