Fast Adaptation Trajectory Prediction Method Based on Online Multisource Transfer Learning

Conventional deep learning-based trajectory prediction methods always adopt offline training based on trajectory data collected in known scenes. Despite its high prediction accuracy, it is unable to process trajectory data acquired in real-time, making it non-trivial to adapt to unknown scenes. To mitigate the above problem, an online multi-source transfer learning-based pedestrian trajectory predictor, dubbed OMTL-PTP, is proposed to achieve fast adaptation of trajectory prediction. OMTL-PTP resorts to online transfer learning to transfer trajectory knowledge from multiple source domains to the target domain, enabling the model to learn from the new scene and continuously improve its trajectory prediction ability. Concretely, we propose several base learners with external memory modules to preserve source domain trajectory knowledge for online knowledge transfer. A multi-hop attention mechanism is introduced in each learner to handle the future uncertainty of generated trajectories. To fully utilize the knowledge from multiple source domains, OMTL-PTP leverages ensemble learning to transfer knowledge from multiple base learners in the source domains to the online learner and fine-tunes the online learner in the target domain. Specifically, all base learners are combined to update the online learner, improving its ability to process future arriving samples and adapt to unknown scenes quickly. Qualitative and quantitative evaluations on ETH/UCY indicate the effectiveness of OMTL-PTP in online learning, which is beneficial for deploying trajectory prediction methods on intelligent edge devices. The code will be released at https://github.com/zjrcczu/OMTL-PTP after acceptance. Note to Practitioners —This paper is motivated by the challenge of online sustained trajectory prediction for unmanned autonomous agents, but it also applies to other trajectory prediction tasks, such as intelligent monitoring. Existing approaches always collect trajectory data from different scenes for training, making the model generalize to other scenarios. However, they may suffer from performance degradation since they cannot learn trajectory knowledge from unknown scenes. This paper suggests a new approach by transferring trajectory knowledge from known scenes to unknown scenes and gradually learning from unknown scenes, inspired by online transfer learning. In this paper, we propose a trajectory predictor based on a memory network and introduce the multi-hop attention mechanism to mitigate future uncertainty of trajectory prediction. We then show how the external memory can preserve trajectory knowledge, which facilitates transferring knowledge from source domains to the target domain. Afterward, we train an online trajectory predictor based on online multi-source transfer learning, improving the generalization and adaptability of trajectory prediction models in unknown scenes and facilitating deploying trajectory prediction models in edge devices. This method also applies to other neural network-based regression tasks that require online sustained learning. In future research, we will improve the trajectory prediction performance while maintaining the online learning ability.