A Multimodal Deep Learning Framework for Depression Detection Using Vision Transformers and Large Language Models

This study proposes a novel multimodal deep learning framework for depression detection, integrating visual, audio, and textual data. Using OpenFace and Librosa for feature extraction, the system employs Vision Transformers (ViTs) to model cross-modal dependencies, while a large language model (Qwen3-32B) analyzes transcribed speech for interpretable symptom reasoning. Outputs from both modalities are fused via a confidence-based mechanism. Experimental validation on the LMVD dataset demonstrates superior performance, overcoming the limitations of unimodal approaches.