The rhythms of head, eyes and hands at intersections

In this paper, we study the complex coordination of head, eyes and hands as the driver approaches a stop-controlled intersection. The proposed framework is made up of three major parts. The first part is the naturalistic driving dataset collection: synchronized capture of sensors looking-in and looking-out, multiple drivers driving in urban environment, and segmenting events at stop-controlled intersections. The second part is extracting reliable features from purely vision sensors looking in at the driver: eye movements, head pose and hand location respective to the wheel. The third part is in the design of appropriate temporal features for capturing coordination. A random forest algorithm is employed for studying relevance and understanding the temporal evolution of head, eye, and hand cues. Using 24 different events (from 5 drivers resulting in ~ 12200 frames analyzed) of three different maneuvers at stop-controlled intersections, we found that preparatory motions range in the order of a few seconds to a few milliseconds, depending on the modality (i.e. eyes, head, hand), before the event occurs.