Classification of Five Emotions from EEG and Eye Movement Signals: Discrimination Ability and Stability over Time

This paper explores the discrimination ability and stability of electroencephalogram (EEG) and eye movement signals over time for classifying five emotions: happy, sad, fear, disgust and neutral. We develop a multimodal emotion dataset called SEED-V with 16 subjects. Two classifiers are trained based on the EEG and eye movement signals. Topographic maps are used to depict the neural patterns of EEG signal. The classification result based on EEG, eye movement, and feature level fusion (FLF) reaches the average accuracies of 70.8%, 59.87% and 75.13%, respectively. The experiment result indicates that: a) the EEG and eye movement signals have good discrimination ability for five emotion classification problem; b) the beta and gamma bands of EEG signal have better discrimination ability than the delta, theta and alpha bands; c) the stable neural patterns of different emotions do exist and are common across sessions; and d) the neural pattern of disgust emotion has high gamma response in the frontal area, while fear emotion has low activation at the top of brain in the gamma band.