Hybrid EEG-fNIRS Brain Computer Interface Based on Common Spatial Pattern by Using EEG-Informed General Linear Model

Hybrid brain computer interfaces (BCI) utilizing the high temporal resolution of electroencephalography (EEG) and the high spatial resolution of near-infrared spectroscopy (fNIRS) are preferred over single-modal BCIs. However, due to the large dimensionality of the multi-class statistical features commonly used in fNIRS signals, it is easy to cause overfitting of the EEG-fNIRS hybrid BCI classifier. Therefore, a low-dimensional feature extraction method for fNIRS based on the EEG-informed fNIRS general linear model (GLM) analysis is proposed in this paper. First, a regression coefficient matrix is obtained by using the EEG-informed fNIRS GLM with a time window added, and the common spatial pattern (CSP) features of this regression coefficient matrix are extracted as the fNIRS features. Lastly, the fNIRS features were combined with the CSP features extracted from the optimal narrow band of EEG as hybrid features, and the support vector machine (SVM) method is used to classify the samples with hybrid features. The proposed method was tested on a publicly available motor imagery dataset. The classification accuracy using fNIRS signals alone reached 68.79% (oxygenated hemoglobin) and 68.62% (deoxygenated hemoglobin), and the classification accuracy of combining EEG-fNIRS features reached 79.48%, which was higher than other existing methods using the same dataset. By using this fNIRS feature extraction method, the problem of poor performance of CSP on fNIRS signals is solved, which not only enriches the processing methods of fNIRS signals, but also improves the classification accuracy of hybrid EEG-fNIRS BCI in motor imagery tasks.