A Spontaneous Driver Emotion Facial Expression (DEFE) Dataset for Intelligent Vehicles: Emotions Triggered by Video-Audio Clips in Driving Scenarios

In this article, a new dataset, the driver emotion facial expression (DEFE) dataset for drivers’ spontaneous emotions analysis is introduced. The dataset includes facial expression recordings from 60 participants during driving. After watching a selected video-audio clip to elicit a specific emotion, each participant completed the driving tasks in the same driving scenario and rated his/her emotional responses during the driving processes from the aspects of dimensional emotion method and discrete emotion method. The study also conducted classification experiments to recognize the scales of arousal, valence, dominance, as well as the emotion category and intensity to establish baseline results for the proposed dataset. Furthermore, this paper compared emotion recognition results difference through facial expressions between dynamic driving and static life scenarios. The results showed that dynamic driving and static life datasets were different in emotion recognition results. To further explore the reasons for the difference in emotion recognition results, the analysis from the AU (action unit) presence perspective was studied. The results showed significant differences in the AUs presence of facial expressions between dynamic driving and static life scenarios, indicating that drivers’ facial expressions may be affected by the driving task to influence the recognition of drivers’ emotions through facial expressions. Therefore, to accurately recognize the drivers’ emotions to establish a reliable emotion-aware human-machine interaction system, thereby improving driving safety and comfort, publishing a human emotion dataset specifically for the driver is necessary. The proposed dataset will be publicly available so that researchers worldwide can use it to develop and examine their driver emotion analysis methods. To the best of our knowledge, this is currently the only public driver facial expression dataset.