Hybrid brain-computer interface with motor imagery and error-related brain activity

OBJECTIVE: Brain-computer interface (BCI) systems read and interpret brain activity directly from the brain. They can provide a means of communication or locomotion for patients suffering from neurodegenerative diseases or stroke. However, non-stationarity of brain activity limits the reliable transfer of the algorithms that were trained during a calibration session to real-time BCI control. One source of non-stationarity is the user's brain response to the BCI output (feedback), for instance, whether the BCI feedback is perceived as an error by the user or not. By taking such sources of non-stationarity into account, the reliability of the BCI can be improved. APPROACH: In this work, we demonstrate a real-time implementation of a hybrid motor imagery BCI combining the information from the motor imagery signal and the error-related brain activity simultaneously so as to gain benefit from both sources. MAIN RESULTS: We show significantly improved performance in real-time BCI control across 12 participants, compared to a conventional motor imagery BCI. The significant improvement is in terms of classification accuracy, target hit rate, subjective perception of control and information-transfer rate. Moreover, our offline analyses of the recorded EEG data show that the error-related brain activity provides a more reliable source of information than the motor imagery signal. SIGNIFICANCE: This work shows, for the first time, that the error-related brain activity classifier compared to the motor imagery classifier is more consistent when trained on calibration data and tested during online control. This likely explains why the proposed hybrid BCI allows for a more reliable means of communication or rehabilitation for patients in need.