An Investigation on the Audio-Video Data Based Estimation of Emotion Regulation Difficulties and Their Association With Mental Disorders

It is reported that emotion regulation difficulties (ERD) happen to characterize several mental disorders. This finding is made based on self-report measures. For enhancing the diagnosis of mental disorders, the use of additional modalities such as audio-video recordings has been explored in the literature. In this study, we present a novel attempt to estimate the ERD using audio-video data. The role of ERD estimated using audio-video data in assessing mental disorders has also been investigated. The study is performed on a locally collected dataset which comprises audio-video data and self-reported responses to a set of questionnaires. The audio-video data is recorded while human participants are conversing with an interactive system in the context of a specific questionnaire. For validation purposes, the created dataset has been benchmarked against an existing publicly available corpus. Two different ways are devised and experimented with for estimating ERD. Further, the estimated ERD is used as an intermediate representation for the assessment of the severity level of opted mental disorders, namely major depressive disorder (MDD) and post-traumatic stress disorder (PTSD). Towards validating the efficacy of the proposed audio-video data-based ERD estimation and its impact on the MDD/PTSD severity level estimation, we perform a correlation analysis among them and find it to exhibit similar trends to those obtained using the self-reported data. This demonstrates that ERD can be used as an intermediate representation of audio-video data to estimate the severity level of MDD/PTSD.