Identifying Brain Regions Related to Word Prediction During Listening to Japanese Speech by Combining a LSTM Language Model and MEG

Abstract Recently, a neuroscientific approach has revealed that humans understand language while subconsciously predicting the next word from the preceding context. Most studies on human word prediction have investigated the correlations between brain activity while reading or listening to sentences on functional magnetic resonance imaging (fMRI) and the predictive difficulty of each word in a sentence calculated by the N-gram language model. However, because of its low temporal resolution, fMRI is not optimal for identifying the changes in brain activity that accompany language comprehension. In addition, the N-gram language model is a simple computational structure that does not account for the structure of the human brain. Furthermore, it is necessary for humans to retain information prior to the N-1 word in order to form a contextual understanding of a presented story. Therefore, in the present study, we measured brain activity using magnetoencephalography (MEG), which has a higher temporal resolution than fMRI, and calculated the prediction difficulty of words using a long short-term memory language model (LSTMLM), which is based on a neural network inspired by the structure of the human brain and has longer information retention than the N-gram language model. We then identified the brain regions involved in language prediction during Japanese-language speech listening using encoding and decoding analyses. In addition to surprisal-related regions revealed in previous studies, such as the superior temporal gyrus, fusiform gyrus, and temporal pole, we also found relationships between surprisal and brain activity in other regions, including the insula, superior temporal sulcus, and middle temporal gyrus, which are believed to be involved in longer-term, sentence-level cognitive processing.