An LSTM Network With Neural Plasticity for Driver Fatigue Recognition on Real Roads

Driver fatigue recognition is a highly challenging issue because of the complexity of road conditions, the dynamics of traffic flow, and the differences between drivers. This article proposes a biologically inspired long short-term memory (LSTM) model with neural plasticity (NP-LSTM) to improve the learning and memory ability of the traditional driver fatigue recognition method, thus improving effectiveness and robustness of monitoring and early-warning systems. First, the approximate entropy (ApEn) of the time series of drivers’ operation behaviors and vehicle status is investigated to explore the features of potential irregularity in fatigue-driving behaviors; then, inspired by the plastic learning mechanism of biological neurons, the intrinsic plasticity and synaptic plasticity are embedded into the LSTM neural network to realize the classified storage of complex road patterns, the dynamics of traffic flow, and the memory of drivers’ individual differences; finally, the dropout technology is introduced to further build a “sparse” neural network, which avoids the repeated training of an unchanged neural network under different conditions and enhances the adaptability and generalization of the whole monitoring and early-warning system. Experimental study on real roads is conducted to demonstrate the effectiveness of the proposed method. The results show that the average recognition accuracy is 88.73%, demonstrating a better recognition performance of the proposed method.