Exploring the effect of visual and auditory information in haptic experience

Haptic experience always has a close relationship with visual and auditory modalities, but little research uses both visual and auditory modalities to map haptic perception in tangible interaction. It is essential to understand how to make haptic information harmonious with visual and auditory modalities, and how users describe haptic feedback in physical interface design. In this paper, participants are divided into three groups: auditory-haptic (AH), visual-haptic (VH), and haptic (H) groups to test a sensory integrated prototype in order to explore the effect of visual and auditory information on haptic experience and users’ description patterns. We find that users’ haptic descriptions range from abstract to concrete and are affected by modalities. Moreover, there is a massive difference in the function of visual and auditory influencing the sense of touch. The findings are beneficial to haptic experience design.