模式
计算机科学
模态(人机交互)
情绪分析
人工智能
水准点(测量)
变压器
自然语言处理
视听
可视化
多模式学习
机器学习
语音识别
特征学习
多模态
深度学习
作者
W. Ronny Huang,Weihong Cai,Z. She
出处
期刊:
[Association for Computing Machinery]
日期:2025-12-06
卷期号:: 1-8
标识
DOI:10.1145/3743093.3770944
摘要
Multimodal sentiment analysis (MSA) predicts sentiment scores by learning correlations across various modalities and leveraging the rich information embedded in the features of each modality, in which the text modality is particularly important. Nevertheless, existing text-dominated methods are limited to statically utilizing visual and audio modalities to assist text modality in sentiment inference. These previous studies overlook that the common presence of redundant and conflicting information within and across modalities leads to differences in the auxiliary roles of visual and audio modality when performing cross-modal interactions dominated by text modality, which hinders the improvement of MSA performance. To address this, we propose the Dynamic Text-guided Multi-task Transformer (DTMT). DTMT learns ultra-modality representations through multi-task learning to effectively extract sentiment clues from each modality. Additionally, DTMT utilizes multi-task generation of contribution weights for visual and audio modalities, dynamically adjusting the interaction proportion between the text modality and these two modalities, thereby achieving more effective multimodal fusion. Extensive experiments on three MSA benchmark datasets (MOSI, MOSEI, CH-SIMS) shows that DTMT achieves state-of-the-art performance.
科研通智能强力驱动
Strongly Powered by AbleSci AI