MVFA: A Multi-View Text-Guided Multimodal Fusion LLM Adapter for Sentiment Analysis and Emotion Recognition

作者
Pengfei Shao,Jisheng Dang,Jiawen Fang,Ning Liu,Wencan Zhang,Bimei Wang,Zhao, Jingwen,Jianhuang Lai,Qi Tian,Tat-Seng Chua
出处
期刊:Cornell University - arXiv [Cornell University]
标识
DOI:10.48550/arxiv.2609.06188
摘要

Multimodal sentiment analysis and emotion recognition in conversations demand effective modeling of heterogeneous interactions across textual, acoustic, and visual modalities. Although large language models (LLMs) offer powerful language understanding, adapting them to multimodal affective computing remains challenging: full-model fine-tuning is computationally prohibitive, while many existing lightweight adapters fail to preserve rich textual cues during cross-modal fusion. To address these limitations, we propose the multi-view text-guided multimodal fusion adapter (MVFA), a parameter-efficient framework that augments frozen LLMs with strong multimodal reasoning capability. MVFA first constructs complementary text views via max pooling, mean pooling, and attention pooling; these views then guide cross-modal interactions with audio and visual features. The fused multimodal representations are subsequently compressed into a compact set of learnable pseudo-tokens through an Enhanced Q-Former Fusion Module. Using ChatGLM3-6B-base as the primary backbone, we further validate MVFA on LLaMA2-7B and Qwen3-8B to examine its portability across multiple frozen LLM backbones. MVFA is evaluated on three challenging datasets: CH-SIMS V2.0, MELD, and CHERMA. Experimental results demonstrate that MVFA achieves state-of-the-art performance on key metrics while updating only a small fraction of parameters. Specifically, it attains 84.62\% Acc2 and 84.59\% F1 on CH-SIMS V2.0, 67.36\% Acc and 66.03\% WF1 on MELD, and 74.66\% Acc on CHERMA. These findings establish multi-view text-guided fusion as an effective and scalable paradigm for parameter-efficient multimodal LLM adaptation in affective computing. The code is publicly available at https://github.com/Overwhelm1208/MVFA.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
清脆火龙果完成签到,获得积分10
刚刚
王露发布了新的文献求助10
刚刚
刚刚
清脆大门完成签到,获得积分10
刚刚
文献打人完成签到,获得积分10
刚刚
litn完成签到 ,获得积分10
刚刚
去偷火龙果完成签到,获得积分10
刚刚
如意完成签到,获得积分20
1秒前
小斌完成签到,获得积分10
1秒前
1秒前
1秒前
科研通AI6.2的应助被wwx0201采纳,获得10
2秒前
DD完成签到 ,获得积分10
2秒前
yangyangyang完成签到,获得积分10
3秒前
3秒前
liu完成签到,获得积分10
4秒前
搞怪的水彤完成签到 ,获得积分10
4秒前
4秒前
抓住努力的尾巴完成签到 ,获得积分10
4秒前
英俊的铭的应助被小雨采纳,获得10
4秒前
中中中中中完成签到,获得积分10
4秒前
蔺山河完成签到,获得积分10
4秒前
悦耳的怀寒完成签到,获得积分10
4秒前
aji发布了新的文献求助10
4秒前
小狗黑头完成签到,获得积分10
4秒前
HM发布了新的文献求助10
5秒前
5秒前
俊逸若之完成签到,获得积分10
5秒前
6秒前
6秒前
6秒前
LC发布了新的文献求助10
6秒前
yeziio完成签到,获得积分10
7秒前
科研通AI6.4的应助被很好采纳,获得10
7秒前
小韩同学发布了新的文献求助10
7秒前
Ireliairelia的应助被生动的雅绿采纳,获得50
7秒前
7秒前
7秒前
7秒前
Liyuhong完成签到,获得积分10
8秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
Energie-Contracting 1500
Klimaschutzrecht durch den Energiesektor im Mehrebenensystem – Ein Rechtsvergleich 1000
Oncology Clinical Trials: Successful Design, Conduct, and Analysis, Second Edition 800
Sustainable Engineered Micro/Nanofibers and Hybrid Nanocomposites for Environmental Remediation 400
Ecotourism 400
Liquidus diagram and viscosity estimation in pseudo-binary system Li2B6O10–LiF 300
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8076814
求助须知:如何正确求助?哪些是违规求助? 9494754
关于积分的说明 21383031
捐赠科研通 7644935
什么是DOI,文献DOI怎么找? 3400268
关于科研通互助平台的介绍 2529896
邀请新用户注册赠送积分活动 2438855