Magnetoencephalography (MEG) based non-invasive Chinese speech decoding

Abstract Objective. As an emerging paradigm of brain–computer interfaces (BCIs), speech BCI has the potential to directly reflect auditory perception and thoughts, offering a promising communication alternative for patients with aphasia. Chinese is one of the most widely spoken languages in the world, whereas there is very limited research on speech BCIs for Chinese language. Approach. This paper reports a text-magnetoencephalography (MEG) dataset for non-invasive Chinese speech BCIs. It also proposes a multi-modality assisted speech decoding (MASD) algorithm to capture both text and acoustic information embedded in brain signals during speech activities. Main results. Experiment results demonstrated the effectiveness of both our text-MEG dataset and our proposed MASD algorithm. Significance. To our knowledge, this is the first study on multi-modality assisted decoding for non-invasive Chinese speech BCIs.