AMA-EEG: Adaptive Multimodal Alignment for Cross-Subject EEG Emotion Recognition

Electroencephalography (EEG)-based emotion recognition typically suffers from low signal-to-noise ratios and significant inter-subject variability. While recent cross-subject EEG emotion recognition methods mainly focus on modeling the intrinsic spatiotemporal dynamics of EEG signals in a unimodal manner, they typically overlook the rich semantic context embedded in the emotional stimuli, such as visual frames and their descriptions, which can serve as stable semantic anchors to guide EEG representation learning. To address this, we propose AMA-EEG, a novel two-stage multimodal alignment framework. In the first stage, we perform multimodal alignment by extracting image and text features with a pre-trained vision-language model, which act as semantic anchors for the EEG signals. An entropy-based adaptive aggregation mechanism is introduced to dynamically weight the contributions of visual and textual features based on their predictive confidence. In the second stage, only the aligned EEG encoder is retained to train a lightweight classifier for cross-subject emotion recognition. Extensive experiments on the FACED and SEED datasets show that AMA-EEG achieves accuracies of 78.29% on FACED-2, 61.30% on FACED-9, and 69.45% on SEED, consistently outperforming existing state-of-the-art methods.