Integrating Dual-Stream Cross Fusion and Ambiguous Exclude Contrastive Learning for Enhanced Human Action Recognition

In the field of semi-supervised human action recognition, the effective utilization of both labeled and unlabeled data remains a central and challenging pursuit. To address this issue, we present an innovative framework (DSCF-AEC) that combines a Dual-stream Cross Fusion network (DSCF) with an Ambiguous Exclude Contrastive Learning (AEC) module. Specifically, our Dual-stream Cross Fusion network utilizes the ST-GCN as encoder, independently encoding two augmented versions of the joint and bone streams, which are subsequently cross-fused to achieve enhanced representation. To further bolster the performance, we designed the AEC module. This module constructs a memory bank capable of distinguishing reliable positive and negative samples, while ambiguous samples are excluded. This strategic approach ensures that, through contrastive learning, the model is trained solely on meaningful and trustworthy samples. Extensive experiments on NTU RGB+D and NW-UCLA datasets validate the effectiveness of our approach. The results indicate that, our proposed method significantly outperforms other existing methods.