Intelligence Evaluation Methods for Autonomous Vehicles

The rapid advancement of artificial intelligence has significantly enhanced the intelligence of autonomous vehicles (AVs). However, owing to the complexity of AV behavior and the high dimensionality of driving environments, the objective and practical quantitative evaluation of AV intelligence remains a significant and unresolved challenge. This paper proposes a robust training-based comprehensive evaluation (RTCE) system specifically designed to assess the intelligence of AVs in the time dimension. Beginning with a foundation model, the first generation of AVs is developed by training in the initial naturalistic traffic scenarios. To effectively test the intelligence of the AVs, we propose an adversarial trajectory optimization technique to generate challenging, critical test scenarios that evaluate the learning capabilities of AVs in complex environments. Through robust training in these complex scenarios, the second generation of AVs is obtained. To objectively and effectively quantify the intelligence of AVs, we further propose a comprehensive evaluation metric system encompassing five dimensions and 14 evaluation metrics. The intelligence score of each AV is computed using the objective multi-criteria decision-making approach. The proposed intelligence evaluation method is validated using various self-evolution autonomous driving algorithms. The results demonstrate that the RTCE method can quantitatively and effectively test the intelligence of AVs in a multi-dimensional and automated manner. Furthermore, the proposed method is flexible and generalizable, making it adaptable to different testing platforms and autonomous driving algorithms.