Empowering Virtual Humans’ Emotional Expression in the Metaverse

The Metaverse is defined as an interconnected network of 3D environments that enable multisensory interactions with virtual environments, digital objects, and people. A central component of a Metaverse is the realism of human interaction with the environment and each other, which is achieved through emotional expression. However, there are limitations in realistically representing human emotions and facial expressions in real time through virtual avatars. In this paper, the authors present a research project called Meta-EmoVis aimed at improving the realism of the Metaverse by detecting human facial expressions and translating them into emotional feedback that influences aspects of the virtual avatar and environment. The Emoj tool is used to collect data and map facial expressions to basic emotions, as well as detect the human engagement level. The goal is to create a world of real social interaction and emotional sharing in virtual spaces that are sensitive and adaptive to human emotional and mental states. Preliminary user tests to evaluate the effectiveness of the application are presented.