A Hybrid YOLOv8 and Vision-Language Model Framework for Brain Tumor Localization and Classification
作者
J Sumanth,Reddi China Appala Naidu
出处
期刊:2018 2nd International Conference on Inventive Systems and Control (ICISC)日期:2025-08-12卷期号:: 937-944被引量:2
标识
DOI:10.1109/icisc65841.2025.11188126
摘要
In medical imaging, identifying and categorizing brain tumors is crucial, where accurate localization and se-mantic interpretation of tumor regions can significantly impact clinical outcomes. This paper presents a hybrid framework that integrates object localization with semantic classification using a vision-language model (VLM). We employ the YOLOv8 architecture to detect and localize tumor regions in MRI brain scans, followed by a fine-tuned MedClipmodel that classi-fies the extracted tumor regions into four categories: glioma, meningioma, pituitary tumor, and no tumor. The MedCLIP model is adapted using a curated dataset of cropped tumor regions with clinical labels. The suggested method achieved a mean average precision (mAP@0.5) of 96.8 % for localization and a classification accuracy of 78 % for tumor type identification after being trained and assessed using publicly accessible brain MRI datasets. The approach also incorporates spatial heuristics to enhance detection performance in difficult cases such as pituitary tumors. Experimental results demonstrate that coupling VLMs with precise object localization enables accurate and interpretable tumor classification in MRI scans.