Efficient Driver Anomaly Detection via Conditional Temporal Proposal and Classification Network

Detecting driver inattentive behaviors is crucial for driving safety in a driver monitoring system (DMS). Recent works treat driver distraction detection as a multiclass action recognition problem or a binary anomaly detection problem. The former approach aims to classify a fixed set of action classes. Although specific distraction classes can be predicted, this approach is inflexible to detect unknown driver anomalies. The latter approach mixes all distraction actions into one class: anomalous driving. Because the objective focuses on finding the difference between safe and distracted driving, this approach has better generalization in detecting unknown driver distractions. However, a detailed classification of the distraction is missing from the predictions, meaning that the downstream DMS can only treat all distractions with the same severity. In this work, we propose a two-phase anomaly proposal and classification framework [driver anomaly detection and classification network (DADCNet)] robust for open-set anomalies while maintaining high-level distraction understanding. DADCNet makes efficient allocation of multimodal and multiview inputs. The anomaly proposal network first utilizes a subset of the available modalities and views to suggest suspicious anomalous driving behavior. Then, the classification network employs more features to verify the anomaly proposal and classify the proposed distraction action. Through extensive experiments in two driver distraction datasets, our approach significantly reduces the total amount of computation during inference time while maintaining high anomaly detection sensitivity and robust performance in classifying common driver distractions.