计算机科学
人工智能
稳健性(进化)
约束(计算机辅助设计)
机器学习
多通道交互
模糊性
多模态
推论
情绪分析
编码器
不变(物理)
完整信息
特征(语言学)
缺少数据
代表(政治)
多模式学习
多式联运
对偶(语法数字)
模式治疗法
水准点(测量)
一致性(知识库)
数据挖掘
任务(项目管理)
模式识别(心理学)
作者
Qi Shen,Junchang Xin,Bing Tian Dai,Zhihong Huang,Zhiqiong Wang
标识
DOI:10.1109/tmm.2025.3639945
摘要
Multimodal data holds great potential in enhancing the accuracy of sentiment analysis. However, the multimodal sentiment analysis (MSA) task faces two practical challenges: i) multimodal heterogeneity significantly increases the difficulty of integrating multimodal information, resulting in insufficient multimodal learning; and ii) multimodal incompleteness easily triggers cross-modal inference biases, seriously hindering multimodal understanding. In this paper, we propose a novel multimodal framework namedcross-modal explicit invariancecoordinateddualconstraintreconstruction (Clive-Dcr) to mitigate MSA for heterogeneous incomplete multimodal data. Specifically, we present a language-oriented cross-modal invariance reasoning (LCIR) approach that can alleviate the inter-modal representation gaps under the supervision of fine-grained sentiment information. More importantly, LCIR can explicitly quantify inter-modal consistency and break the vagueness of heterogeneous modal information, thereby fully exploring the complementary advantages between invariant features and specific features. In the prediction head, we develop a multimodal purification learning mechanism to minimize the intervention of sentiment-irrelevant information on MSA. To enhance the robustness under incomplete modalities, we design a dual constraint reconstruction (DCR) strategy. During the reconstruction of missing features, DCR not only performs high-level feature distribution alignment but also extends the cross-modal invariance reasoning from the complete to the incomplete multimodal scenarios. Comprehensive experiments on three multimodal datasets verify that our Clive-Dcr achieves remarkable performance under complete and incomplete multimodal patterns.
科研通智能强力驱动
Strongly Powered by AbleSci AI