Hybrid 3D Vision Transformer Model for Parkinson’s Disease Diagnosis

Parkinson’s disease (PD) is a rapidly progressing neuro-degenerative disorder, and its early detection is a very challenging task. As a crucial modality for PD detection, 3D structural Magnetic Resonance Imaging (MRI) images have been widely adopted in deep learning methods. However, conventional 3D Convolutional Neural Networks (CNNs), despite their promising performance, lack the capacity to capture long-range dependencies. Meanwhile, although pre-trained large-scale Vision-Language Models (VLMs), like BioMedCLIP, offer powerful generalization capabilities and excel in modeling long-range dependencies due to their Transformer-based architecture, they face challenges in adapting to 3D medical data because of spatial information loss in 2D projections. To overcome these limitations, we propose a novel adaptive fine-tuning framework, namely Hybrid3D-ViTA, which integrates a lightweight 3D ResNet-10 for local spatial feature extraction with the pre-trained visual encoder of BioMedCLIP for global context modeling. Additionally, we introduce a CT Adapter for feature alignment and a Patch Attention module for discriminative region enhancement. Experiments conducted on the PPMI dataset demonstrate that our method achieves state-of-the-art performance in PD diagnosis.