Noncontact Human-Machine Interaction for Air-Writing Based on Piezoelectric Micromachined Ultrasonic Transducers

With rapid development of Internet of Things (IoT) and virtual reality (VR), sensors based on human-machine interaction (HMI) are crucially required to have attributes of natural and intuitive operation experience using various adaptable and hygienic noncontact techniques. In this study, we develop a real-time noncontact HMI system for air-writing based on piezoelectric micromachined ultrasonic transducers (pMUTs), which have great advantages including easy integration, miniaturization, low power consumption, and less influences by environmental factors such as light or sound. These pMUTs often have problems of weak signals and limited measurement range for complex gesture recognitions. To address these issues, we propose a machine learning (ML) algorithm which effectively fuses multiple features of time-of-flight (ToF), voltage amplitudes, and echo energies, and significantly increases the detection range and accuracy of pMUTs for arbitrary gesture recognition. We demonstrate a real-time computer control to search and browse websites simply using only one finger air-writing with a recognition accuracy of 96.62% for 16 gestures of characters (including numbers, letters, and symbols) within a distance range of 15 cm. We believe our method revolutionizes functionalities and adaptabilities of noncontact HMI in VR, smart home/vehicle, smart cities, and healthcare.