Lightweight CNN-LSTM gesture recognition using synchronized sEMG and A-mode ultrasound

Hybrid surface electromyography (sEMG) and A-mode ultrasound (AUS) sensing can provide complementary information for hand gesture recognition. sEMG reflects electrical muscle activation, whereas AUS captures morphology-related tissue displacement. This study adapts a compact CNN-LSTM framework for recognizing nine hand gestures from synchronized sEMG and AUS recordings. The processed dataset used for the proposed model contains eight participants, four sEMG channels, four AUS channels, and 85,359 paired windows. Each window contains four AUS frames and temporally aligned sEMG samples, producing inputs of size [4, 4, 50] for sEMG and [4, 4, 128] for AUS. The model uses modality-specific convolutional frame encoders, two bidirectional LSTM branches, attention pooling, gated feature fusion, and a compact classifier. On an 80/20 per-subject per-gesture chronological split performed before overlapping-window generation, the model achieved 96.35% accuracy, 96.53% macro precision, 96.51% macro recall, and 96.50% macro F1-score with 271,411 trainable parameters. Under split-before-windowing evaluation, the CNN-LSTM outperformed the evaluated single-modality and multimodal CNN, LSTM, and Transformer baselines while retaining a modest state-dictionary size of 1.037 MB. Subject-independent and recording-held-out evaluation remains necessary before deployment claims can be made.