“What” and “when” predictions jointly modulate speech processing

Adaptive behavior rests on predictions based on statistical regularities in the environment. Such regularities pertain to stimulus contents (“what”) and timing (“when”), and both interactively modulate sensory processing. In speech streams, predictions can be formed at multiple hierarchical levels of contents (e.g. syllables vs. words) and timing (faster vs. slower time scales). Whether and how these hierarchies map onto each other remains unknown. Under one hypothesis, neural hierarchies may link "what" and "when" predictions within sensory processing areas: with lower vs. higher cortical regions mediating interactions for smaller vs. larger units (syllables vs. words). Alternatively, interactions between "what" and "when" regularities might rest on a generic, sensory-independent mechanism. To address these questions, we manipulated “what” and “when” regularities at two levels – single syllables and disyllabic pseudowords – while recording neural activity using magnetoencephalography (MEG) in healthy volunteers (N=22). We studied how neural responses to syllable and/or pseudoword deviants are modulated by “when” regularity. “When” regularity modulated “what” mismatch responses with hierarchical specificity, such that responses to deviant pseudowords (vs. syllables) were amplified by temporal regularity at slower (vs. faster) time scales. However, both these interactive effects were source-localized to the same regions, including frontal and parietal cortices. Effective connectivity analysis showed that the integration of “what” and “when” regularity selectively modulated connectivity within regions, consistent with gain effects. This suggests that the brain integrates “what” and “when” predictions that are congruent with respect to their hierarchical level, but this integration is mediated by a shared and distributed cortical network. Significance statement This study investigates how the brain integrates predictions about the content (“what”) and timing (“when”) of sensory stimuli, particularly in speech. Using magnetoencephalography (MEG) to record neural activity, researchers found that temporal regularities at slower (vs. faster) time scales enhance neural responses to unexpected disyllabic pseudowords (vs. single syllables), indicating a hierarchical specificity in processing. Despite this specificity, the involved brain regions were common across different hierarchical levels of regularities, and included frontal and parietal areas. This suggests that the brain uses a distributed and shared network to integrate “what” and “when” predictions across hierarchical levels, refining our understanding of speech processing mechanisms.