Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts: Design and Evaluation

Augmented reality (AR) interaction methods are leaning towards more natural techniques, such as voice commands, hand gestures, and in-air tapping for input. From a security perspective, however, recent works have demonstrated that these methods, such as in-air tapping, are vulnerable to inference attacks where an adversary is capable of reconstructing input in the virtual environment using low-level hand-tracking data with high accuracy. This paper addresses the defense of in-air tapping mechanisms against inference attacks by developing and evaluating a 3D curved keyboard for input. Our design exploits the symmetry between the virtual and physical worlds enabling the inference attack in the first place and increasing the uncertainty of the adversary by manipulating the geometric aspects of this keyboard plane in 3D. We evaluate our design through numerous experiments and show it to be robust against inference attacks, where the adversary’s accuracy in obtaining the correct input text is reduced to 0% (from 87%) and at most to just 18% within the top-500 candidate reconstructions.