Deep Stroop: Using eye tracking and speech processing to characterize people with neurodegenerative disorders while performing the Stroop Test

Abstract Although many neurodegenerative diseases affect different neural circuits, they often express complex and overlapping symptom profiles making them difficult to differentiate precisely. Current methods of analyzing patients are limited to bedside examination, patient self-rating scales, semiquantitative clinician-rating scales, and other observational evidence, which are often non-specific, resulting in open multiple interpretations and ambiguity in diagnosis and treatment plans. We present a method to analyze patient symptom profiles using multimodal analysis of subjects performing the Stroop Test. We use high-sample-rate eye tracking and speech recording tools to record subject behavior while completing the Stroop Test and simultaneously analyze multiple traits of their interaction with the test. We compare the performance of healthy controls to patients with Parkinson’s Disease, Alzheimer’s Disease, and other neurodegenerative diseases with clinical parkinsonism. We automatically extract metrics based on eye motor behavior, gaze characteristic uttered responses, and the temporal relationship between gaze and uttered responses. We identify many that have clinical relevance through high correlations with existing MoCA and MDS-UPDRS, many of which have significantly different distributions between groups. We present here our analysis approach, provide freely available source code to replicate it and demonstrate the potential of multi-modal recording and analysis of patients throughout their execution of neuro-psychological tests like the Stroop Test.