EEG emotion recognition via Identity based Multi-gate Mixture-of-Experts network

Empowering computer systems to automatically recognize human emotions has become an urgent need in the field of human-computer interaction (HCI). Two-dimensional emotion (Valence-Arousal) models are commonly used to represent emotions. Up to now, the correlation between emotion dimensions has rarely been investigated, and subject-independent EEG emotion recognition is still a challenging task. For this purpose, we introduce multi-task learning (MTL) into EEG emotion recognition. MTL learns different emotion dimensions simultaneously and extracts correlation information between dimensions in task-sharing space to coordinate the optimization of multiple emotion dimensions. We further propose Identity based Multi-gate Mixture-of-Experts (IDMMOE), which allocates part of model subspace for each subject in a customized manner according to the subject's identity. Extensive experiments were conducted on DEAP dataset. Three MTL models were implemented: Shared-Bottom, Multi-gate Mixture-of-Experts, and Customized Gate Control respectively. They were compared with a single-task learning model trained separately on valence and arousal. Experimental results demonstrate that two emotion dimensions are intrinsically related, and MTL acquires such correlation information and improves prediction accuracy in both emotion dimensions. In addition, IDMMOE achieves average accuracies of 89.5% and 89.7% for valence and arousal respectively and it is effective for subject-independent experiment.