Affordances for Multi-device Gestural Interactions in Augmented Reality

This thesis research explores augmented reality (AR) gestural interface design for multi-device controls (e.g. smart lights, smart speakers, computer displays, tablets, etc.) This design problem encounters three significant challenges. First, AR gestural interface design is still an emerging technology, with which most users are not familiar. It, therefore, requires the design must communicate intuitively even to novice users. Second, people have various mental models of interacting with different devices, and the design needs to keep their mental model consistent. Third, gestural interfaces initially hold various constraints, such as inaccurate tracking, lack of tactile feedback, and more easily to cause user fatigue.

To address all those problems, this thesis adopts the existing theory of affordances to AR interaction design and explores the new potential design paradigms for future AR gestural interface. The study leverages a photo diary study for user inquiry, and designs 24 working prototypes for user testing and iterating. Ultimately I proposes five design recommendations that can inspire future work in this domain. The recommendations include a five-element framework for managing perceived affordances in AR, a conceptual model of hand-space and world-space interaction, harnessing metaphors and physical affordances for multi-device controls, and empowering the abstract with embodiment & materialization.