IGTPT: Intent-Caption Guided Trajectory Prediction Transformer

Trajectory prediction is crucial for autonomous vehicles to make safe and informed decisions. However, the lack of transparency in current trajectory prediction models introduces significant security risks, because their output contains almost no explanatory details. To address these challenges and bridge this research gap, we propose a novel approach IGTPT that not only predicts vehicle trajectories but also generates textual descriptions of the vehicle’s intent. The vehicle’s intent can be broadcast to nearby vehicles to enhance decision-making transparency and increase user trust. Our approach directly confronts the opaque nature of existing systems by providing clear, understandable explanations for autonomous decisions, thereby enhancing both the security and reliability of these systems. We enhanced the BDD-X dataset to create the BDD-XE, a specialized dataset for trajectory prediction and behavior description, which our framework uses to achieve superior results in both prediction accuracy and behavioral interpretation compared to established baseline methods. We demonstrate the practical applicability of our framework through a complete system that processes past raw driving videos and trajectory observations to deliver real-time predictions along with insightful behavioral narrations and reasoning.