Deep Learning Based Take-Over Performance Prediction and Its Application on Intelligent Vehicles

Take-over performance plays a significant role in evaluating drivers' state, and serves as a crucial reference for enhancing control transitions in the context of conditionally automated driving. In this study, we aim to predict minimum anticipated collision time (min ACT), an indicator of drivers' take-over performance, in expectation of promoting safer take-overs via deep learning, so that drivers' state detriment of take-over safety could be adjusted accordingly with intelligent human-machine interaction algorithms predictably. By incorporating multi-source information including drivers' state, drivers' demographics, surrounding traffic features as well as driver-vehicle interaction characteristics, network model “ACTNet” was proposed to facilitate continuous estimation. Depthwise separable convolution and non-local self-attention were utilized to prevent overfitting and establish spatial dependency over fixation heatmap, respectively. To overcome data distribution imbalance, class balanced loss was used in conjunction with regression loss to realize more accurate predictions. Driving simulator experiment was conducted with dataset collected for the subsequent verification of the proposed algorithm. Potentialities of deep learning methods were highlighted for take-over studies, contributing to the design of intelligent human-machine interaction systems in conditional automation. Our findings present a valid method of deep learning in predicting drivers' take-over performance and meanwhile have implications for the development of intelligent adaptive take-over time budget regulation and dynamic drivers' state adjustment algorithms.