Robust Optical Based Hand Interaction for Virtual Reality

Using optical sensors to track hand gestures in virtual reality (VR) simulations requires issues such as occlusion, field-of-view, accuracy and stability of sensors to be addressed or mitigated. We introduce an optical hand-based interaction system that comprises two Leap Motion sensors mounted onto a VR headset at different orientations. Our system collects sensor data from the leap motions, combines and processes it to produce optimal hand tracking data, that minimises the effect of sensor occlusion and noise. This contrasts with previous systems that do not use multiple head-mounted sensors or incorporate hand-data aggregation. We also present a study that compares the proposed system with glove-based and traditional motion controller-based interaction. We investigate hand interactions effect on the feeling of naturalness and immersion. The results show that the use of two head-mounted sensors and the data aggregation system increased the number of valid hands presented to the user and can be successfully applied to VR. The user study shows that there is a strong preference for the proposed system due to the natural feeling and freeing interaction. The absence of an indirect interface such as gloves or controllers was found to aid in creating a more natural and immersive experience.