Design of an Efficient Multimodal Deep Learning Framework for Assessing Mental Workload Using Eye Tracking and Physiological Parameters

An important predictor of human cognitive and physical performance, it is necessary to precisely and efficiently measure mental workload for applications ranging from individualized health care to productivity enhancement. Present methods rely mainly, and separately, on physiological measurements or eye-tracking data, critically limiting the precision with which mental workload can be accurately assessed across levels of memory, response time, and precision. The present findings point to the need for an integrated, multimodal strategy to surmount these limitations, and significantly improve mental workload estimation accuracy. As a result, here we propose a novel multimodal deep learning architecture that effectively incorporates eye-tracking and physiological data. Apart from complex information related to fixation time, saccade velocity, and averaged pupil diameter from eye movement data, our methodology captures a range of physiological signals, such as ECG readings, glucose fluctuations, and blood pressure changes. An accurate assessment of mental demands is then made by fusing data from multiple sources using ensemble learning and an efficient 1D Convolutional Neural Network (1D CNN) classifier. The proposed model outperformed previous techniques with 2.9%, 3.5%, and 3.4% increases in precision, accuracy, and recall, respectively. The fact that the methodology also demonstrated a 2.5% drop in latency levels further reinforced the promise of a faster implementation of the approach for a more responsive, real-time mental workload estimation technique. The current study establishes the groundbreaking potential of our multimodal approach in providing a thorough and accurate assessment of mental burdens, thereby opening up important applications in a wide variety of domains.