Capsule neural networks on spatio-temporal EEG frames for cross-subject emotion recognition

Scalp EEG plots are plots of scalp potentials against time, and hence, capture spatial information, owing to the placement of electrodes on the scalp, as well as, temporal information from variations in brain waves. In this paper we propose a novel method to make a combined representation of spatial and temporal information, by incorporating the signals into a sparse spatio-temporal frame, such that it can be easily processed by deep learning algorithms in the computer vision domain. Familiarities of a model to the test data in the setting of emotion recognition from EEG, is also defined, and a form of data splitting such that the model has to perform on a set with which it has the minimum degree of familiarity is introduced. A CapsNet architecture is trained on DEAP dataset to perform on a cross-subject binary classification task, and tuning of the hyperparameters using Bayesian Optimization is analyzed. The proposed model reports a best-case accuracy of 0.85396 and average case accuracy of 0.57165 for LOO subject, and a best case of 1.0 and average case of 0.51071 for unseen-subject-unseen-record classification, when averaged across all the classes (i.e., valence, dominance, arousal, and liking), which is comparable to that reported by other works.