Symmetry-Driven hyper feature GCN for skeleton-based gait recognition

• SDHF-GCN is the first work that models the skeleton-based graph structure with graph-based neural network in gait recognition task. • The symmetry perceptual principle enforces the dependencies between related joints and suppress the noises caused by joint estimation. • Hierarchical features can improve the expressive and discriminative power of the gait features. • SDHF-GCN renders substantial improvements over mainstream methods, especially in the coat-wearing scenario. • Silhouette-based and skeleton-based action patterns are certified to be highly complementary. Gait recognition, as an attractive task in biometrics, remains challenging due to significant intra-class changes of clothing and pose variations across different cameras. Recent approaches mainly focus on silhouette-based gait mode, which is easy to model in Convolutional Neural Networks (CNNs). Compared with silhouettes, the dynamics of skeletons essentially convey more robust information, which is invariant to view and clothing changes. Conventional approaches for modeling skeletons usually rely on hand-crafted features or traversal rules, thus resulting in limited expressive power and difficulties of generalization. In this work, we address the skeleton-based gait recognition task with a novel Symmetry-Driven Hyper Feature Graph Convolutional Network (SDHF-GCN), which goes beyond the limitations of previous approaches by automatically learning multiple dynamic patterns and hierarchical semantic features in a unified Graph Convolutional Network (GCN). This model involves three dynamic patterns: natural connection, temporal correlation and symmetric interaction, which enriches the description of dynamic patterns by exploiting symmetry perceptual principles. Furthermore, a hyper feature network is proposed to aggregate the hierarchical semantic features, including dynamic features at the high level, structured features at the intermediate level, and static features at the low level, which complement each other to enhance the discriminative ability. By integrating different patterns in the hierarchical structure, the model is able to generate versatile and discriminative representations, thus improving the recognition rate. On the CASIA-B and OUMVLP-Pose datasets, the proposed SDHF-GCN renders substantial improvements over mainstream methods, especially in the coat-wearing scenario, with superior robustness to covariate factors.