TCTR: Text-Guided Contrastive Learning with Token-Level Reconstruction Network for missing modalities in multimodal sentiment analysis

计算机科学 安全性令牌 模式 人工智能 自然语言处理 情绪分析 社会科学 计算机安全 社会学
作者
Zhihao Yang,Qing He,MingHao Yu,Nisuo Du,Yijie Lu
出处
期刊:Information Fusion [Elsevier BV]
卷期号:126: 103571-103571 被引量:5
标识
DOI:10.1016/j.inffus.2025.103571
摘要

Multimodal sentiment analysis (MSA) tasks in incomplete multimodal data scenarios must account for random missing or noisy interference of modality information, aiming to perform robust sentiment analysis on multimodal data. This also reflects the trend of MSA tasks transitioning from idealized laboratory settings to real-world conditions, making it a current research hotspot in multimodal learning. However, existing studies still face limitations in missing modeling analysis, and lacking effective modeling of missing scenarios. Moreover, current methods primarily focus on completing missing modality features in the feature space, overlooking information supplementation in the semantic space, which is crucial for multimodal sentiment analysis tasks. To address this, we propose a text-guided fine-grained network model: Text-Guided Contrastive Learning with Token-Level Reconstruction Network (TCTR). This is motivated by the fact that the text modality typically contains more direct and complete sentiment information. In TCTR, we first design the Token-level Missing Inspection (TMI) module to perform token-level missing modeling on the guided modality, addressing the limitation of insufficient capture of critical sentiment information in missing inspection through fine-grained missing analysis. Subsequently, in the Semantic Contrastive Learning for Missing Modality Supplementation (SCL-MMS) module, we leverage constructed negative sample labels to jointly complete missing sentiment information from both the feature space and the semantic space, mitigating the issue of inadequate supplementation quality caused by relying solely on the feature space in existing methods. Finally, building on prior research, we perform interaction and fusion of multimodal features to enable sentiment polarity prediction. Through performance comparisons with state-of-the-art methods and ablation studies on various datasets, the experimental results demonstrate that TCTR achieves superior sentiment polarity prediction across different modality-missing scenarios, effectively enhancing the robustness of MSA tasks in such conditions. • MSA tasks under missing modality scenarios. • Missing modality inspection and sentiment supplementation. • Token-level missingness inspection and fine-grained missingness modeling. • Missing modality supplementation guided by label semantics and feature space. • The effectiveness of TCTR is validated through extensive experiments.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
清爽的以晴完成签到,获得积分20
1秒前
Akim应助雾昂采纳,获得10
1秒前
2秒前
laser完成签到,获得积分10
2秒前
Xuan完成签到,获得积分10
3秒前
liu.lzy应助zhangxiaopan采纳,获得30
3秒前
gafssr发布了新的文献求助10
4秒前
4秒前
4秒前
4秒前
科目三应助繁荣的代玉采纳,获得10
5秒前
laser发布了新的文献求助10
5秒前
5秒前
哇哈哈哈发布了新的文献求助10
6秒前
6秒前
6秒前
7秒前
Shirky发布了新的文献求助10
7秒前
7秒前
脱羰甲酸发布了新的文献求助10
8秒前
zepenta发布了新的文献求助10
8秒前
zjy147发布了新的文献求助10
9秒前
Ava应助魔幻冷风采纳,获得10
9秒前
9秒前
9秒前
五五发布了新的文献求助10
10秒前
marry完成签到,获得积分10
10秒前
顺利凡松发布了新的文献求助10
10秒前
重要帆布鞋应助fengxin采纳,获得30
10秒前
ht发布了新的文献求助10
11秒前
11秒前
Army616发布了新的文献求助10
11秒前
zhouxw27完成签到,获得积分10
12秒前
在水一方应助牧童采纳,获得10
13秒前
精明纸鹤应助Lijunjie采纳,获得10
14秒前
大个应助lyzzz采纳,获得10
14秒前
Miao完成签到,获得积分10
14秒前
田様应助可乐鸡翅采纳,获得10
15秒前
16秒前
风起人散完成签到,获得积分10
17秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
The Organometallic Chemistry of the Transition Metals 800
Chemistry and Physics of Carbon Volume 18 800
The Organometallic Chemistry of the Transition Metals 800
The formation of Australian attitudes towards China, 1918-1941 640
Signals, Systems, and Signal Processing 610
全相对论原子结构与含时波包动力学的理论研究--清华大学 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 工程类 有机化学 化学工程 生物化学 计算机科学 物理 内科学 复合材料 催化作用 物理化学 光电子学 电极 细胞生物学 基因 无机化学
热门帖子
关注 科研通微信公众号,转发送积分 6439728
求助须知:如何正确求助?哪些是违规求助? 8253611
关于积分的说明 17567315
捐赠科研通 5497817
什么是DOI,文献DOI怎么找? 2899368
邀请新用户注册赠送积分活动 1876189
关于科研通互助平台的介绍 1716646