EEGMoE: A Domain-Decoupled Mixture-of-Experts Model for Self-Supervised EEG Representation Learning

Existing deep learning models for electroencephalogram (EEG) are typically tailored for specific tasks, datasets, or even subjects. This specialization restricts their applicability, reducing both perceptual capabilities and overall generalizability. Recent research has begun to explore large-scale pretraining for EEG representation learning. These efforts often focus on enabling data from different domains to be encoded by the same model through unifying the data format. However, this approach tends to overlook the importance of decoupling EEG representations across various domains, leading to the missing of valuable domain-specific details. Given the diverse distribution of large-scale EEG data forming multiple distinct domains, we hope to learn not only domain-shared representations but also to learn domain-specific representations. Therefore, in this article, we propose EEG mixture of experts (EEGMoE), a domain-decoupled self-supervised pretraining model for EEG representation learning. Specifically, EEGMoE introduces a Transformer-based domain-decoupled encoder, featuring specific and shared expert groups within our mixture-of-experts (MoE) block. The specific expert group adopts Top- $K$ routing to select the most appropriate $K$ experts for each token, while the shared expert group employs soft routing, leveraging all experts to learn for each token. Pretrained on various EEG datasets from multiple tasks, EEGMoE is then fine-tuned and validated on new datasets covering three mainstream EEG tasks, including emotion recognition (ER), motor imagery (MI) classification, and mental workload detection. Experimental results show that EEGMoE outperforms state-of-the-art models on three public datasets, demonstrating its strong generalization ability to new domains. Extensive experiments and visualizations further highlight the importance and effectiveness of disentangling domain-specific representations. Our code and model will be released.