Knowledge distillation based lightweight domain adversarial neural network for electroencephalogram-based emotion recognition

Individual differences in Electroencephalogram (EEG) could cause domain shift which would significantly degrade the accuracy of cross-subject emotion recognition. To tackle this issue, domain adversarial neural networks (DANN) are adopted to deal with domain shift. However, if the feature extractor within DANN is cumbersome, the limited quantity of EEG data may result in overfitting and negative transfer. In this work, we propose a knowledge distillation (KD) based DANN to obtain a reliable lightweight feature extractor and improve domain-invariant feature learning. The proposed method contains two stages, and temporal-spatial feature interaction is adopted throughout two stages. In the feature-based KD framework, a transformer-based hierarchical temporal-spatial learning model is served as the teacher model. The student model, which is a lightweight version of the teacher model, is composed of Bi-LSTM units. Furthermore, the student model could be supervised to learn robust feature representations of the teacher model by leveraging complementary latent temporal and spatial features. In the DANN-based cross-subject emotion recognition, the obtained student model and a lightweight temporal-spatial feature interaction module are combined as the feature extractor. Then, the aggregated temporal-spatial features are fed to the emotion classifier and domain classifier for domain-invariant feature learning. To validate the effectiveness of proposed method, we conduct experiments on DEAP dataset, focusing on arousal and valence classification with subject-independent strategy. The outstanding performance and t-SNE feature visualization could provide evidence of the effectiveness. Besides, the proposed method has achieved a greater improvement than the teacher-based DANN in the domain-invariant learning. This result indicates that the proposed method could effectively alleviate the negative transfer problem.