Lighter and faster: A multi-scale adaptive graph convolutional network for skeleton-based action recognition

In recent years, graph convolutional network (GCN) has gained significant popularity in skeleton-based action recognition due to its ability to effectively model non-Euclidean data. However, most existing high-accuracy GCN-based models often utilize deep neural networks with numerous layers, resulting in increased computational costs. To address this issue, we propose a lightweight-modified multi-scale adaptive graph convolutional network (LMA-GCN) for skeleton-based action recognition, which can efficiently capture relationships between distant joints in the human skeleton and consider the uniqueness of different data samples as well while maintaining high inference speed and low complexity. In addition, a novel lightweight metric λlw is presented for effective evaluation of the model’s comprehensive performance between accuracy and lightweight. A simplified skeleton sequence representation is also presented for skeleton-based action recognition. Extensive experiments demonstrate the excellent comprehensive performance of the model LMA-GCN on three large public datasets: NTU RGB+D 60, NTU RGB+D 120, and UAV-Human. LMA-GCN obtains comparable accuracy with only 0.13M parameters and its inference speed reaches 80.2 sequences/second on one RTX 3060 GPU, which provides a simple, effective and feasible method for meeting the needs of “connected for anything, anywhere, anytime” and portable devices in internet of things (IoT) technology.