计算机科学
模式
编码器
水准点(测量)
人工智能
多模式学习
多模态
情绪分析
视听
机器翻译
模态(人机交互)
光学(聚焦)
自然语言处理
机器学习
多媒体
地理
社会学
大地测量学
万维网
物理
光学
操作系统
社会科学
作者
Bo Yang,Bo Shao,Lijun Wu,Xiaola Lin
标识
DOI:10.1016/j.neucom.2021.09.041
摘要
Multimodal Sentiment Analysis (MSA) is a challenging research area that investigates sentiment expressed from multiple heterogeneous sources of information. To integrate multimodal information including text, visual and audio modalities, state-of-the-art models focus on developing various fusion strategies, such as attention and outer product. However, the inferior quality of visual and audio features that is commonly observed in this area has not aroused much attention. We argue that this issue will obstruct the performance of the fusion strategies to a considerable extent. Therefore, in this paper, we propose Multimodal Translation for Sentiment Analysis (MTSA), a multimodal framework that improves the quality of visual and audio features by translating them to text features extracted by Bidirectional Encoder Representations from Transformers (BERT). Experiments on two benchmark datasets CMU-MOSI and CMU-MOSEI show that our model performs better than the state-of-the-art methods on both datasets across all the metrics, which illustrates the effectiveness of our method.
科研通智能强力驱动
Strongly Powered by AbleSci AI