CMGNN: Cross-Modal Emotion Recognition via EEG-Face Alignment and Expert-Guided Fusion

Emotion recognition from multimodal data remains challenging due to the semantic gap and temporal-spatial misalignment between EEG signals and facial expressions. To address this, we propose a cross-modal framework that integrates EEG and facial features via modality-guided semantic representation learning. Temporal features are extracted by stacked MAMBA-based blocks capturing long-range dependencies. A Cross-Modal Scaling and Shifting (CMSS) mechanism uses EEG features to refine and align facial representations, reducing modality discrepancies. The fused features pass through a GRUcontrolled Mixture-of-Experts (MoE-GRU) module, where a learnable gating network dynamically selects specialized Transformer experts. This combination effectively handles misalignment and enables dynamic feature fusion, enhancing recognition accuracy and robustness. Experiments on DEAP and MAHNOBHCI demonstrate state-of-the-art results, validating its real-world applicability in affective computing.