Effects of Noise and Reward on Pupil Size and Electroencephalographic Speech Tracking in a Word‐Detection Task

ABSTRACT Speech is hard to understand when there is background noise. Speech intelligibility and listening effort both affect our ability to understand speech, but the relative contribution of these factors is hard to disentangle. Previous studies suggest that speech intelligibility could be assessed with EEG speech tracking and listening effort via pupil size. However, these measures may be confounded, because poor intelligibility may require a larger effort. To address this, we developed a novel word‐detection paradigm that allows for a rapid behavioural assessment of speech processing. In this paradigm, words appear on the screen during continuous speech, similar to closed captioning. In two listening experiments with a total of 51 participants, we manipulated intelligibility by changing signal‐to‐noise ratios (SNRs) and modulated effort by varying monetary reward. Increasing SNR improved detection performance along with EEG speech tracking. Additionally, we find that pupil size increases with increased SNR. Surprisingly, when we modulated both reward and SNR, we found that reward modulated only pupil size, whereas SNR modulated only EEG speech tracking. We interpret this as the effects of arousal and listening effort on pupil size and of intelligibility on EEG speech tracking. The experimental paradigm developed here may be beneficial when assessing hearing devices in terms of speech intelligibility and listening effort.