FMASketch: Freehand Mid-Air Sketching in AR

AbstractAbstractSketching is a common way to depict design ideas freely. Augmented reality technology extends the human–computer interaction space from the screen to the real world, providing a new application space for 3D sketching. However, due to the difficulty of depth perception, it is difficult for sketching strokes to maintain good planar and spatial consistency, making drawing 3D sketches in AR challenging. In this article, we propose FMASketch—a method of drawing mid-air sketches similar to building blocks. The auxiliary surfaces can ensure that the strokes drawn by users can always stay on the same plane as drawing in 2D space. Gestures are used to stitch together multiple drawing planes, and the strokes on the planes form a 3D sketch, which is very similar to how 3D modeling is done on the desktop by drawing different views and assembling them. Experiments have verified the effectiveness of the auxiliary surface generation method and gesture operations in this article. The mid-air sketching method is easy to learn and provides a new idea for rapid modeling in AR scenarios.Keywords: Mid-air sketchingARauxiliary drawing surfacegestures Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationNotes on contributorsYang ZhouYang Zhou is currently pursuing a doctoral degree in Software Institute of Nanjing University. He received his master's degree in Software Engineering at Nanjing University in 2022. His research interests include interaction in Augmented Reality and sketch-based content generation.Peixin YangPeixin Yang is currently a master student in Software Institute, Nanjing University. He received his bachelor's degree in Software Engineering at Nanjing University in 2022. His research interests are interaction in Augmented Reality.Xinchi XuXinchi Xu is currently a master student in Software Institute, Nanjing University. He received his bachelor's degree in Software Engineering at Dalian University of Technology University in 2020. His research interests include 3D sketching in Virtual Reality.Bingchan ShaoBingchan Shao is currently a master student in Software Institute, Nanjing University. She received her bachelor's degree in Information Engineering at Tianjin University of Finance and Economics in 2020. Her research interests are interaction design for children.Guihuan FengGuihuan Feng is currently working as an associate professor of Software Institute, Nanjing University. In 2009, she got her Ph.D. degree in Computer Science and Technology Department, Nanjing University. Her research interests include VR/AR interaction, pen-based interaction, software usability, AIGC etc.Jie LiuJie Liu is currently working as an associate research fellow at Institute of Software, Chinese Academy of Sciences. In 2016, he got his Ph.D. degree at the Chinese Academy of Sciences. His research interests include intelligent perception, brain-like multimodal information intelligence processing, and big data intelligence analysis.Bin LuoBin Luo is currently working as a professor at the Software Institute, Nanjing University. His research interests lie in Intelligent operating Systems and databases and Human-computer interaction.