Identifying Stable EEG Patterns in Manipulation Task for Negative Emotion Recognition

Negative emotion recognition during manipulation task plays crucial role in human-machine interaction, where diverse cognitive variables coexist and influence each other. However, traditional emotion experiments often overemphasize emotion induction while overlooking other practical cognitive tasks, which leads participants to suffer from simplistic emotional experiences and ultimately compromises the real-world applicability of the emotional data collected. To incorporate critical cognitive variables into emotion elicitation, we utilize joystick-based real-time emotion annotation to encourage subjects to continuously feel emotional intensity, to advisedly decide when to manipulate the joystick, and to physically operate it. Consequently, at least two essential cognitive variables—decision-making and action—are integrated into emotion perception. Following this, we develop a novel negative emotion dataset called CRED, which includes a variety of physiological data, particularly Electroencephalograph (EEG). To assess the stability of emotional EEG patterns, we employ strict statistical analysis and a dual-branch transformer (DBT) with the gradient-based attribution method on the proposed CRED. Additionally, two well-known public datasets (SEED and SEED-V) are used to verify the DBT. Compared to traditional methods, DBT improves classification accuracy by approximately 5% on CRED and by around 2% on the public datasets. The experimental results indicate that the occipital lobe plays a crucial role in the discrimination of negative emotions; the critical frequency bands vary between the five emotions in the CRED. Specifically, the low-delta rhythm is associated with anger, while fear is influenced by both theta and alpha rhythms; disgust is found to be significant in the theta rhythm; and for neutral emotions, both low-delta and alpha rhythms are identified as crucial. In summary, our findings demonstrate the existence of stable emotional EEG patterns when additional cognitive variables are involved.