Investigating speech features and automatic measurement of cognitive load

The ability to measure cognitive load level in real time is extremely useful for improving the efficiency of interfaces and contents delivering, especially when interfaces and contents get complex in a multimedia environment. Speech is highly suitable for measuring cognitive load due to its non-intrusive nature and ease of collection. In this paper, we investigated the patterns of prosodic features and confirmed it is relevant to cognitive load. We also explored varied classification techniques to capture those relevant patterns of speech features. Gaussian Mixture Model (GMM), Support Vector Machine (SVM), and a hybrid SVM-GMM based classifiers were investigated with MFCC and pitch features. Individual systems and a fusion based system were evaluated on two different task scenarios - reading comprehension and Stroop test. The SVM-GMM based system achieved the highest performance on both tasks and improved the accuracy of three levels classification to 75.6% and 82.2%, respectively.