Convolutional Neural Networks-Motivated High-Performance Multi-Functional Electronic Skin for Intelligent Human-Computer Interaction

A multi-functional electronic skin (e-skin) with high sensitivity and wide response range is crucial for intelligent human-computer interaction (HCI) system. Here, a multi-functional hybrid e-skin (TPES) integrated by a piezo-capacitive pressure sensor (PCPS) and a triboelectric nanogenerator (TENG) is reported. An ion-gel electrospinning doped with MXene is combined with a micro-structured ion-gel to enrich the structural variations of the overall device, enabling TPES to achieve a high sensitivity of 616.42 kPa−1, a broad response range of 0.6 Pa–1 MPa, and a fast response time of 5.6 ms. The vast surface area of the mixed cellulose foam layer effectively enhances the electrical output of the TENG, making TPES’s application scenarios become more flexible. By further integrating the different functionalities of the TPES with one-dimensional convolutional neural networks and HCI module, a gesture recognition and material perception HCI system is realized. Additionally, benefiting from the excellent performance of the PCPS and TENG, a multi-information recognition HCI system is established, capable of simultaneously recognizing the pressure and surface material of the balls and enabling real-time interaction with a virtual character. In summary, the proposed work offers a novel avenue for practical intelligent HCI system in virtual reality field.