A Depression Detection Auxiliary Decision System Based on Multi-Modal Feature-Level Fusion of EEG and Speech

By improving the accuracy of depression recognition and designing a consumer-oriented depression detection system, consumers are expected to receive convenient and fast e-health services. Recently, depression recognition methods based on the analysis of physiological and behavioral data have attracted attention. In particular, the research on Electroencephalography (EEG) and speech signals becomes hotspots. However, EEG is susceptible to individual differences, while speech signal is susceptible to environmental factors. In this study, we propose an auxiliary decision-making system for depression detection that considers both physiological and behavioral factors by fusing EEG and speech signals. Compared to existing studies, our proposed multi-modal fusion strategy exploits more linear and nonlinear features to support the recognition of task classifications. In addition, we analyze the functional connectivity of brain regions to facilitate EEG feature extraction. Considering the non-stationary feature of EEG and speech signals, we perform filtering, artifact processing, and time-frequency domain processing. Furthermore, we integrate the EEG and speech signals at the feature level and train their classification. Performance evaluation results show that our proposed multi-modal feature fusion strategy achieves 86.11% accuracy on the dataset of major depressive disorders, and 87.44% recognition accuracy on the healthy controls.