sEMG-Based Gesture Recognition Using Sigimg-GADF-MTF and Multi-Stream Convolutional Neural Network

To comprehensively leverage the temporal, static, and dynamic information features of multi-channel surface electromyography (sEMG) signals for gesture recognition, considering the sensitive temporal characteristics of sEMG signals to action amplitude and muscle recruitment patterns, an sEMG-based gesture recognition algorithm is innovatively proposed using Sigimg-GADF-MTF and multi-stream convolutional neural network (MSCNN) by introducing the Sigimg, GADF, and MTF data processing methods and combining them with a multi-stream fusion strategy. Firstly, a sliding window is used to rearrange the multi-channel original sEMG signals through channels to generate a two-dimensional image (named Sigimg method). Meanwhile, each channel signal is respectively transformed into two-dimensional subimages using Gram angular difference field (GADF) and Markov transition field (MTF) methods. Then, the GADF and MTF images are obtained using a horizontal stitching method to splice these subimages, respectively. The Sigimg, GADF, and MTF images are used to construct a training and testing dataset, which is then imported into the constructed MSCNN model for experimental testing. The fully connected layer fusion method is utilized for multi-stream feature fusion, and the gesture recognition results are output. Through comparative experiments, an average accuracy of 88.4% is achieved using the Sigimg-GADF-MTF-MSCNN algorithm on the Ninapro DBl dataset, higher than most mainstream models. At the same time, the effectiveness of the proposed algorithm is fully verified through generalization testing of data obtained from the self-developed sEMG signal acquisition platform with an average accuracy of 82.4%.