Mid-air gestures for in-vehicle media player: elicitation, segmentation, recognition, and eye-tracking testing

Abstract There was a problem of great distraction in traditional touch interactive operation of in-vehicle information interface, but the application of gesture recognition technology on in-vehicle system improved this problem. However, users had less experience in this new interactive mode, and the cognitive deviation of the agreement between gestures and commands could directly affect the safety of drivers. The main purpose of this paper was to obtain the user's preference mid-air gestures for the in-vehicle information interface by using the user-elicitation method. In addition, the optimized gesture recognition network structure was applied to the prototype system of in-vehicle information interaction with user-defined mid-air gestures developed by us, and the effectiveness of the system was evaluated by collecting eye movement indicators of users through the simulation of driving eye movement experiments. In the process of user elicitation, the principle of command prompt and agreement rate was introduced. According to the agreement rate elicited by users (AR = .397), we got the gesture consensus set design direction of in-vehicle information control. The experimental results of the eye movement index showed that the method of user-defined mid-air gestures can effectively improve driving safety on in-vehicle media control, and reduced the distraction of users when driving compared with the traditional touch-based method. Article highlights The user's preferred in-vehicle media control task gestures were obtained through the user elicitation method. Constructed an in-vehicle secondary task control prototype system based on gesture recognition. The eye movement experiment proved that the prototype system can reduce the user's distraction when operating the in-vehicle's secondary tasks.