计算机科学
情绪分析
多模态
情态动词
人工智能
多通道交互
自然语言处理
情报检索
人机交互
万维网
化学
高分子化学
作者
Shunxiang Zhang,Jiajia Liu,Yixuan Jiao,Yulei Zhang,Lei Chen,Kuan‐Ching Li
摘要
User-generated multimodal data can provide powerful sentiment clues for sentiment analysis task. Existing works have aligned common sentiment features in different modalities through various multimodal fusion methods. However, these works have certain limitations: (1) Previous research works only align common sentiment features between image and text, without fully exploring interactions among these features, leading to suboptimal analysis results. (2) Redundant noise in image and text increases the risk of feature misalignment during cross-modal alignment. To address these issues, we propose a Multimodal Semantic Fusion Network (MSFN) to deeply explore the semantic relationship between image and text for Multimodal Sentiment Analysis (MSA). Specifically, we align image region and text word features related to sentiment by using a gated attention mechanism. Subsequently, we employ graph convolutional networks to model the interactions among these features to obtain explicit sentiment semantics. The proposed gated attention mechanism corrects potential feature misalignment during cross-modal alignment using a gating mechanism. Moreover, considering not all image–text pairs have explicit corresponding sentiment features, we integrate implicit sentiment semantics to our model for enhancing reliability in analysis. Experimental results on benchmark datasets demonstrate the effectiveness of our proposed model compared to baselines.
科研通智能强力驱动
Strongly Powered by AbleSci AI