Decoding throat-language using flexibility sensors with machine learning

Throat vibration signals contain potential information for communication. However, there is little systematic research on throat vibration signals at present. Here, we propose a novel throat-language decoding system (TLDS) for capturing signals of throat vibration aided by flexible, low-cost and self-powered sensors and semantic analysis with a machine learning classifier. A polyvinylidene fluoride (PVDF) flexible piezoelectric sensor was prepared to collect the throat vibration signals. The sensor has a safe human skin fit, high softness, excellent response repeatability, outstanding linear sensitivity, and long-term durability. After denoised, the time-frequency dynamics features and nonlinear dynamics features of the throat vibration signals were extracted, and the Grid Search-Support Vector Machine (GS-SVM) was applied to recognize the features. TLDS has achieved satisfactory results in a series of tasks, including letters recognition, speaker recognition, and semantic recognition. The average accuracy of single-person letters recognition was 90.55%, even with multi-person, the accuracy rate was still up to 87.26%. Besides, the accuracy of speaker recognition and simple semantic recognition were 95.97%, and 97.50%, respectively. Our work provides a promising approach that can provide unparalleled value in helping people who cannot speak to live a convenient life with accessible communications.