Adversarial traffic scene generation considering harm, rarity, and ambiguity for autonomous driving testing

• A universal adversarial testing framework is designed. • The aim is to generate worst-case traffic scenarios. • The assessment considers harm, ambiguity, and rarity. • The method is assessed based on real-world driving data. Given that autonomous vehicles operate in open-world, safety-critical environments, it is essential to rigorously assess their reliability, particularly under worst-case traffic scenarios, to ensure passenger safety. Most existing testing methods fail to adequately evaluate the impact of adversarial behaviors and neglect the compound errors introduced when the subject under test is incorporated into the test scene. To expedite testing and reveal potential vulnerabilities in AV algorithms, we propose a universal adversarial testing framework designed to generate worst-case traffic scenarios focused on prediction and planning, assessed from harm, ambiguity, and rarity perspectives. For harm, we apply noncooperative game theory to strategically disrupt the tested vehicle while ensuring the disruptions remain reasonable via an asymmetric risk field. For rarity and ambiguity, we encourage the adversarial agents to exhibit high levels of aleatoric and epistemic uncertainty by maximizing the k-nearest neighbor distance in the latent space of a surrogate predictor, thereby crafting conditions that diverge from conventional scenarios in the training set. Our adversarial traffic scene generation algorithm is evaluated on the Argoverse 2 dataset and further validated on the NGSIM dataset without requiring retraining. Through comparison with other testing methods and comprehensive ablation studies, we qualitatively and quantitatively demonstrate that our algorithm effectively, efficiently, and reasonably produces highly critical traffic scenarios for interactive AV planning, including optimization-based and learning-based autonomous driving algorithms.