Event-Driven Edge Deep Learning Decoder for Real-Time Gesture Classification and Neuro-Inspired Rehabilitation Device Control

Assistive neuro-inspired rehabilitation devices are essential for people who have suffered a spinal cord injury, stroke, or limb amputation in their activities of daily living. Neuro-inspired rehabilitation devices typically use a single-modal biosignal with a conventional machine-learning algorithm on an embedded edge device for gesture classification. Although deep learning decoders provide high-accuracy gesture classification, the mismatch in the computational complexity and resource availability of edge devices has limited the deployment of real-time gesture inference on embedded devices. In this study, we describe an event-driven, edge-compatible deep neural network (DNN) capable of classifying gestures from a single or hybrid biosignal detected at the edge. The DNN-based decoders were deployed on a field-programmable gate array (FPGA) to classify motor intent acquired from the biosensors for intuitive control of a 3D-printed upper limb rehabilitation device. The study was validated with thirty-three subjects offline and on-device. Offline average classification accuracy of 93.14% for single-modal electromyography (EMG-Net), 50.42% for single-modal electroencephalography (EEG-Net), and 93.35% for hybrid-modal biosignal (Hybrid-Net) using the 8-bit fixed-point quantization-aware method were obtained, while the real-time inference on the FPGA resulted in 94.97%, 58.27%, and 92.73%, respectively. The EMG biosensor shifted 5 cm to examine model degradation yielded 11.5% and 2.64% accuracy loss for the on-device EMG-Net and Hybrid-Net. The event-driven algorithm implemented performed with a reliability of 1, ensuring inference with voluntary gesture grasp. The study reports that hybrid biosignals outperformed single-modal EEG in gesture classification offline and on-device and single-modal EMG in case of EMG electrode shift. Additionally, this article demonstrates an end-to-end approach that deploys a DNN decoder to an edge device for neuro-inspired control of the dexterous hand devoid of an Internet of Things (IoT) connection. The data and code are available at the following repository: https://github.com/HumanMachineInterface/Gest-Infer.