Spatial and Temporal Awareness Network for Semantic Segmentation on Automotive Radar Point Cloud

Radar sensors are vital for autonomous driving due to their consistent and dependable performance, even in challenging weather conditions. Semantic segmentation of moving objects in sparse radar point clouds is an emerging task that contributes to improving the safety of autonomous driving. However, the methods still need to be explored since radar points in driving scenes are distributed sparsely and irregularly. Typical methods address the sparsity by aggregating multi-scan data to generate dense point clouds, where the temporal correlation is ignored. Inputting consecutive frames of point clouds can preserve temporal information, but the irregular distribution makes it difficult to achieve interframe communications. In this article, we propose a scheme to process points of multi-scan data into a single frame with the availability of temporal features. Our novel network, called Spatial and Temporal Awareness Network (STA-Net), enables points at different times to interact and establish their spatiotemporal connections to comprehend the surroundings of these points. Furthermore, we design a shallow feature extraction method based on the radar measurements of points, which enhances the representation of local features. To further improve the capability of the network to distinguish between various moving objects, we also introduce a prompt layer inspired by prompt-based learning. This layer instructs the network to generate a discriminative representation for each type of moving object. Our experiments demonstrate that our network achieves state-of-the-art performance compared to other methods designed on the RadarScenes dataset. In particular, it shows a remarkable ability to segment small objects such as pedestrians.