Causal Confusion in Pedestrian Crossing Intention Prediction for Autonomous Vehicles: The Role of Ego-Vehicle Speed

Pedestrian crossing intention prediction is crucial for autonomous vehicles due to their inherent inertia, yet challenging. The prevailing practice is to leverage multi-modal data that correlate with pedestrian crossing intention as input to infer it, with ego-vehicle speed being a commonly used modality. However, from a causal perspective, we identify two critical issues overlooked by existing methods: 1) The causal relationship between ego-vehicle speed and pedestrian crossing intention is inconsistent across the training and testing phases, leading to a distribution shift in the data; 2) There exists a bidirectional causality between ego-vehicle speed and pedestrian crossing intention, comprising forward causality and anti-causality. The imbalanced distribution of these two causal directions in natural datasets results in causal confusion, further exacerbating the distribution shift. These issues lead to a counter-intuitive hypothesis: removing ego-vehicle speed as input can actually benefit prediction performance. Therefore, we propose LIM (Less Is More), a uni-modal model that utilizes only skeleton sequences as input. LIM features specially designed modules for efficient skeleton sequence processing, eliminating the reliance on unstable ego-vehicle speed. Moreover, LIM employs adversarial training to identify and remove any latent ego-vehicle speed information embedded within the skeleton sequences. LIM achieves competitive prediction performance compared to state-of-the-art multi-modal models while offering superior real-time capabilities and lower computational costs. By revealing critical issues overlooked by most existing work and providing a more robust crossing intention prediction model, our work contributes to the development of safer autonomous vehicles.