A robust neural prosthetic control strategy against arm position variability and fatigue based on multi-sensor fusion

Abstract Objective. Multi-modal sensor fusion comprising surface electromyography (sEMG) and A-mode ultrasound (US) has yielded satisfactory performance in gesture recognition, aiding amputees in restoring upper limb function. However, prior research conducted in laboratory settings with consistent arm positions lacks practical application for amputees using prostheses. Additionally, motion tests utilized in current studies necessitate prolonged gesture execution, while constant muscle contractions introduce fatigue and increase misclassification risk in practical applications. Consequently, implementing a robust control is imperative to mitigate the limitations of constant arm positions and muscle contractions. Approach. This paper introduces a novel decoding strategy for online applications based on A-mode US, sEMG, and inertial movement unit (IMU) sensor fusion. The decoding process comprises four stages: arm position selection, sEMG threshold, pattern recognition, and a post-processing strategy, which preserves the previous short-duration hand gesture during rest and aims to improve prosthetic hand control performance for practical applications. Main results. The offline classification accuracy achieves 96.02% based on fusion sensor decoding. It drops to 90.72% for healthy participants when wearing an arm fixture that simulates the load of a real prosthesis. The implementation of the post-processing strategy results in a 92.51% online classification accuracy (ONCA) for recognized gestures in three varied arm positions, significantly higher than the 78.97% ONCA achieved when the post-processing strategy is disabled. Significance. The post-processing strategy mitigates constant muscle contraction, demonstrating high robustness to prosthetic hand control. The proposed online decoding strategy achieves remarkable performance on customized prostheses for two amputees across various arm positions, providing a promising prospect for multi-modal sensor fusion based prosthetic applications.