Self-Supervised Spatiotemporal Graph Neural Networks With Self-Distillation for Traffic Prediction

Spatiotemporal graph neural networks (GNNs) have been used successfully in traffic prediction in recent years, primarily owing to their ability to model complex spatiotemporal dependencies within irregular traffic networks. However, the feature extraction processes in these methods are limited in their exploration of the inner properties of traffic data. Specifically, graph and temporal convolutions are local operations and can hardly utilize information from wider ranges, which may affect the long-term prediction performance of such methods. Furthermore, deep spatiotemporal GNNs easily suffer from poor generalization owing to overfitting. To address these problems, this study presents a novel traffic prediction method that integrates self-supervised learning and self-distillation into spatiotemporal GNNs. First, a self-supervised learning module is used to explore the knowledge from the input data. An auxiliary task based on temporal continuity is designed to capture the contextual information in traffic data. Second, a self-distillation framework is developed as an implicit regularization approach that transfers knowledge from the model itself. The combination of self-supervision and self-distillation further mines the knowledge from the data and the model, and the generalization ability and stability of the prediction model can be improved. The proposed model achieved superior or competitive results compared with several strong baselines on six traffic prediction datasets. In particular, the maximum performance improvement ratios for the six datasets were 3.0% (MAE), 5.2% (RMSE), and 3.8% (MAPE). These results demonstrate the effectiveness of the proposed method.