GLF-STAF: A Global-Local-Facial Spatio-Temporal Attention Fusion Approach for Driver Emotion Recognition

Driver emotion recognition is crucial for improving human-vehicle interaction and driving safety. Existing studies often neglect the temporal dynamics of facial expressions and rarely consider combining global and local facial features. To address these limitations, this paper proposes a global-local-facial spatio-temporal attention fusion (GLF-STAF) approach for driver emotion recognition using video data. The method captures both global and local facial expression changes while integrating spatial and temporal facial features. The proposed framework comprises three key stages. First, global facial keypoints and local facial regions are extracted from consecutive video frames. To reduce redundant information and improve data discriminability, a multi-class joint spatial filter is applied to the extracted keypoints. Second, a spatio-temporal parallel framework is employed to effectively utilize both spatial and temporal features. Third, a gated fusion block is incorporated to dynamically integrate spatial and temporal information flows. Additionally, to enhance the data distinction among different emotion classes, an angular center loss function with an additive gap cost is designed. Experimental results demonstrate that the GLF-STAF approach achieves satisfactory performance in driver emotion recognition. This approach has the potential to enhance the capabilities of onboard driver monitoring systems, contributing to advancements in consumer electronics and in-vehicle technology.