Validity and reliability of self‐reported and neural measures of listening effort

Abstract Listening effort can be defined as a measure of cognitive resources used by listeners to perform a listening task. Various methods have been proposed to measure this effort, yet their reliability remains unestablished, a crucial step before their application in research or clinical settings. This study encompassed 32 participants undertaking speech‐in‐noise tasks across two sessions, approximately a week apart. They listened to sentences and word lists at varying signal‐to‐noise ratios (SNRs) (−9, −6, −3 and 0 dB), then retaining them for roughly 3 s. We evaluated the test–retest reliability of self‐reported effort ratings, theta (4–7 Hz) and alpha (8–13 Hz) oscillatory power, suggested previously as neural markers of listening effort. Additionally, we examined the reliability of correct word percentages. Both relative and absolute reliability were assessed using intraclass correlation coefficients (ICC) and Bland–Altman analysis. We also computed the standard error of measurement (SEM) and smallest detectable change (SDC). Our findings indicated heightened frontal midline theta power for word lists compared to sentences during the retention phase under high SNRs (0 dB, −3 dB), likely indicating a greater memory load for word lists. We observed SNR's impact on alpha power in the right central region during the listening phase and frontal theta power during the retention phase in sentences. Overall, the reliability analysis demonstrated satisfactory between‐session variability for correct words and effort ratings. However, neural measures (frontal midline theta power and right central alpha power) displayed substantial variability, even though group‐level outcomes appeared consistent across sessions.