Let IoT Know You Better: User Identification and Emotion Recognition Through Millimeter-Wave Sensing

Emotion recognition, particularly contactless recognition via wireless sensing, has shown its promise in diverse applications. However, the previous works only focus on emotions rather than the person, i.e., the premise is already knowing who the subject is, without considering the issue of identifying subjects. We envision that user identification and emotion recognition together will bring more adaptive and personalized Internet of Things applications, e.g., a smart home system can react to specific emotions of a specific user, independently. In this work, we move forward to investigate the problem of simultaneous user identification, using only physiological indicators embedded in wireless signals reflected off from targets. Toward the objective, in this article, we first carry out a comprehensive measurement study, which validates the feasibility of simultaneous user identification and emotion recognition. Moreover, the measurement also discovers that the key challenge lies in the limitation of artificial features and the substantial emotion feature deviation across different days, which hinders accurate and robust sensing. To resolve the challenge, we design two multiscale neural networks, incorporated with a custom-built feature attention mechanism, so as to obtain rich feature expression and, thus, enhance the important features for accurate recognition. We prototype mmEMO using a commercial off-the-shelf millimeter-wave radar and experimental evaluation shows that mmEMO can achieve 87.68% user identification accuracy and 80.59% emotion recognition accuracy, respectively.