The role of auditory and visual cues in the interpretation of Mandarin ironic speech

Irony is a form of non-literal language. Its decoding may require receivers to use information from both the auditory (e.g., prosody) and visual (e.g., facial expressions) modalities. However, little is known about the role of cues in different modalities in understanding irony. Therefore, the present study examined the relative contributions of auditory and visual cues in the interpretation of Mandarin ironic speech. Sixty-three native Mandarin speakers participated in a perception experiment in which two types of irony (ironic criticisms and ironic compliments) and their literal counterparts (sincere criticisms and sincere compliments) were presented in audio-only, visual-only, and audio-visual conditions. A two-alternative forced-choice task was adopted to measure the listeners’ interpretation of ironic expressions. Ironic criticisms and ironic compliments were interpreted best in the visual-only condition as compared to the other conditions. Furthermore, the participants could employ auditory cues alone to accurately interpret ironic criticisms (accuracy significantly above chance), but not ironic compliments. This study demonstrated a dominant role of visual information in interpreting Mandarin ironic speech, while the role of auditory information is contingent on the type of irony.