Effect of audio-visual interaction on soundscape in the urban residential context: A virtual reality experiment

This research aims to study the independent and interaction effects of aural and visual indicators on soundscape descriptors in the residential area. A virtual reality (VR) experiment was conducted to reproduce audio-visual stimuli and to collect subjective data using the questionnaire. Typical visual elements and aural environment in residential areas were reproduced using Unity software. Ten audio-only stimuli and 42 audio-visual stimuli (combination of 14 visual stimuli and three aural stimuli) were evaluated for various soundscape descriptors by 32 normal-hearing participants. There are four research findings. Firstly, a visual distraction was found on the effect of crowd sound on pleasantness. Secondly, the independent effects of indicators were analyzed. The sound of heavy traffic negatively affects pleasantness, while no significant difference has been found between no traffic and light traffic conditions. The crowd sound and PgVI (Playground View Index) positively affect eventfulness, while roughness and GVI (Green View Index) have a negative effect on eventfulness. Thirdly, traffic sound was found as a moderator of other indicators’ effect on pleasantness and eventfulness. For instance, GVI (Green View Index) can positively affect pleasantness in no traffic and light traffic conditions, while GVI does not affect pleasantness in heavy traffic conditions. Lastly, the present study did not find calmness correlated with eventfulness in residential areas, contrary to the previous studies. A possible explanation for this result might be that the correlation between descriptors is context-dependent.