Spatiotemporal Predictive Models for Irregularly Sampled Time Series

To perform the long-term spatiotemporal sequence prediction (SSP) task with irregular time sampling assumptions, we build the sequence-to-sequence models based on the Trajectory Gated Recurrent Unit (TrajGRU) network and our proposed deep learning modules. First, we design a novel attention mechanism, namely Motion-based Attention (MA), and insert it into the TrajGRU network to create the TrajGRU-Attention model. In particular, the TrajGRU-Attention model can alleviate the impact of the vanishing gradient, which leads to the blurry effect in the long-term predictions and handle irregularly sampled time series. Second, leveraging the advances in Neural Ordinary Differential Equation (NODE) technique, we propose the TrajGRU-Attention-ODE model, which can be applied in continuous-time applications. To evaluate the performance of the proposed models, we select four available spatiotemporal datasets with increasing complexity levels, including the MovingMNIST, MovingMNIST++, KTH Action, and TAASRAD19. Our models outperform the state-of-the-art NODE model and generate better results than the standard TrajGRU model for SSP tasks with different types of time sampling.