ST-PCT: Spatial–Temporal Point Cloud Transformer for Sensing Activity Based on mmWave

The millimeter-wave (mmWave) spectrum has become a core of wireless communication, which has the advantages of richer spectrum resources, larger communication bandwidth, and smaller spectrum interference. Human activity recognition (HAR) by mmWave radar based on point cloud attracts significant attention due to its nature of privacy-preserving, which is an important task of realizing integrated sensing and communication (ISAC). This article proposes a framework of spatial–temporal point cloud transformer (ST-PCT) to realize high precision of HAR, based on sequential point cloud after preprocessing from mmWave radar without voxelization. In ST-PCT, it consists of four enhanced components: 1) a framewise spatial neighbor embedding module to extract the local feature; 2) a temporal and spatial attention mechanism module to find connections within and across frames; 3) an optimized attention mechanism to improve the efficiency of feature extraction; and 4) a sensor fusion module with more motion information to improve the difference between activities. We experimentally evaluate the efficiency of our framework compared with several approaches based on the voxelization or point cloud directly. The experimental results have demonstrated that the proposed ST-PCT network greatly outperforms the other approaches in terms of overall accuracy (oAcc), achieving 99.06% and 99.44%, respectively, on two data sets.