A High‐Accuracy Facial Expression Recognition System Combining Triboelectric Hydrogel Sensors With Deep Learning

Abstract Facial expression recognition (FER) is significant for daily mental health monitoring because facial expressions reflect an individual's mental condition. However, it is still hard to achieve accurate and convenient FER using wearable devices. Here, a high‐accuracy, self‐powered, and intelligent FER system is reported consisting of a triboelectric hydrogel sensor network to collect facial expression signals and a deep learning model to process and recognize the signals. The triboelectric hydrogel sensors are demonstrated to show excellent properties, such as 50% stretchability, 90% transparency, and a response time of 48 ms. With a 1D convolutional neural network, six basic expressions can be recognized with an average recognition accuracy of 99.44%. Finally, a 3D virtual character model is built on a computer to display real emotions synchronously. Compared with previous similar reports, this system can recognize more types of expressions with significantly improved accuracy. Therefore, this work can potentially not only help doctors to determine a patient's mental health condition through virtual communication while protecting the patient's privacy but also provide a highly promising approach for virtual telemedicine in the future.