Detection of potential anxiety in social media based on multimodal fusion with deep learning methods

The global prevalence of anxiety disorders is the highest among mental disorders in 2020. However, most people still ignore the danger of anxiety disorders and most of the research on mental disorders only focuses on depression patients. Therefore, this paper makes a Multi-Modal-Anxiety(MMA) dataset for anxiety disorder detection based on data from Weibo social media, and proposes a Multimodal-Anxiety-Detection Network(MADNet) which fused three dimensions: textual information, image information and behavior information. The model maps textual features and non-textual features into the same semantic space for fusion via Multimodal-Anxiety-Information fusion method(MAI) to predict the anxiety tendency for a single post. The experimental results show that the model has achieved F1-score 70.96% and AUC-ROC 70.91% on the MMA dataset, which is state-of-the-art among the existing models. This paper also explores and analyses the prediction of the model through interpretable methods to prove the validity of the model. Overall, this paper provides a usable dataset, model baseline, and multimodal fusion methods for further research on anxiety disorder based on social media. The code associated with this paper is available at https://github.com/Shuzhong-Lai/MADNet.