Stylus and Gesture Asymmetric Interaction for Fast and Precise Sketching in Virtual Reality

AbstractThis research investigates fast and precise Virtual Reality (VR) sketching methods with different tool-based asymmetric interfaces. In traditional real-world drawing, artists commonly employ an asymmetric interaction system where each hand holds different tools, facilitating diverse and nuanced artistic expressions. However, in virtual reality (VR), users are typically limited to using identical tools in both hands for drawing. To bridge this gap, we aim to introduce specifically designed tools in VR that replicate the varied tool configurations found in the real world. Hence, we developed a VR sketching system supporting three hybrid input techniques using a standard VR controller, a VR stylus, or a data glove. We conducted a formal user study consisting of an internal comparative experiment with four conditions and three tasks to compare three asymmetric input methods with each other and with a traditional symmetric controller-based solution based on questionnaires and performance evaluations. The results showed that in contrast to symmetric dual VR controller interfaces, the asymmetric input with gestures significantly reduced task completion times while maintaining good usability and input accuracy with a low task workload. This shows the value of asymmetric input methods for VR sketching. We also found that the overall user experience could be further improved by optimizing the tracking stability of the data glove and the VR stylus.Keywords: Human-centered computinghuman-computer interaction (HCI)virtual reality immersive sketching Disclosure statementNo potential conflict of interest was reported by the author(s).Notes1 https://www.tiltbrush.com/2 https://www.gravitysketch.com/3 https://www.vive.com/us/accessory/controller/4 https://www.oculus.com/accessories/quest/5 https://www.logitech.com/en-eu/promo/vr-ink.html6 https://www.logitech.com/en-us/promo/vr-ink.html7 https://stretchsense.com/8 https://stretchsense.com/9 https://openbrush.itch.io/openbrushAdditional informationNotes on contributorsQianyuan ZouQianyuan Zou is a PhD candidate specializing in XR experiences for design. With a keen focus on VR and AR applications, he is currently researching asymmetric interfaces for immersive design at the University of Auckland’s Empathic Computing Lab under the guidance of Professor Mark Billinghurst and Dr. Allan Fowler.Huidong BaiHuidong Bai obtained his PhD from the HIT Lab NZ in 2016, under the supervision of Prof. Mark Billinghurst and Prof. Ramakrishnan Mukundan. He is presently a Research Fellow at the Empathic Computing Laboratory at The University of Auckland. His research focuses on remote collaborative Mixed Reality (MR) and empathic interfaces.Lei GaoLei Gao, a PhD graduate from HitLab NZ, University of Canterbury, is now a software engineer at Determ. His research centers on large-scale remote collaboration through AR interfaces with scene scanning and capture tech. Gao brings extensive expertise in computer vision-based AR tracking, reconstruction, and rapid 3D data rendering.Gun A. LeeGun A. Lee Senior Lecturer, specializes in interactive techniques for sharing virtual experiences in AR and immersive 3D environments. Currently focused on enhancing remote collaboration through AR and wearables, his research aims to enrich communication cues and broaden participation in larger groups.Allan FowlerAllan Fowler Senior Lecturer, held tenured positions in Game Design and Software Engineering at renowned institutions, including Columbia University and UC Santa Cruz. Research areas include gamified learning, CS Education, Game Jams, and Human-Computer Interaction, with a global perspective influenced by New Zealand’s Manawatu-Whanganui region.Mark BillinghurstMark Billinghurst holds a PhD in electrical engineering from the University of Washington, supervised by Prof. T. Furness III and Prof. L. Shapiro. He is presently a Professor heading the Empathetic Computing Laboratory at Auckland Bioengineering Institute, The University of Auckland.