A Survey on Datasets for the Decision Making of Autonomous Vehicles

Autonomous vehicles (AVs) are expected to reshape future transportation systems, and decision making is one of the critical modules toward high-level automated driving. To overcome those complicated scenarios that rule-based methods could not cope with well, data-driven decision-making approaches have aroused more focus. The datasets to be used in developing data-driven methods dramatically influence the performance of decision making; hence, it is necessary to have a comprehensive insight into the existing datasets. From the aspects of collection sources, driving data can be divided into vehicle-, environment-, and driver-related data. This study compares the state-of-the-art datasets of these three categories and summarizes their features, including sensors used, annotation, and driving scenarios. Based on the characteristics of the datasets, this survey also discusses potential applications of datasets on various aspects of AV decision making, assisting researchers in finding appropriate ones to support their own research. The future trends of AV dataset development are summarized.