Pedestrian Crossing Intention Prediction From Surveillance Videos for Over-the-Horizon Safety Warning

Pedestrian crossing intention prediction could effectively prevent traffic injuries and improve pedestrian safety. This paper focuses on pedestrian crossing intention prediction from surveillance cameras, which could provide over-the-horizon safety warnings and has the potential to better ensure pedestrian safety, compared with that from on-board cameras. However, most prediction-based methods are designed with a fundamental assumption that the visual data is collected from an on-board camera rather than a bird-eye-view one, thus the prevalent methods in this research domain do not match surveillance scenarios. To deal with this issue, an automated learning framework is proposed, in which a pedestrian-centric environment graph is primarily constructed to reflect visual variations and spatiotemporal relationships between pedestrians and their surroundings. After that, a Graph Convolutional Network (GCN) based environment encoder and a pedestrian-state encoder are designed to extract prominent environment features and pedestrian behavior features, respectively. Finally, an intention prediction decoder is developed to extrapolate the probability of crossing intention. Experimental results demonstrate that each component in the framework contributes to performance improvement and their combination obtains state-of-the-art performance, suggesting the effectiveness and superiority of our framework.