RemoPhysEmo: A Multimodal Emotion Dataset with Remotely Sensed Physiological Signals via Video and Radar

计算机科学 人工智能 计算机视觉 雷达 遥感 雷达跟踪器 雷达成像 特征(语言学) 合成孔径雷达 模式识别(心理学)
作者
Zheng Gao,Xiaobai Li,葛介艺,Hao Zou,Jing Zheng,Li Li,Feng Lin,Jingwen Feng,Ying Chen,Youchen Luo
出处
期刊: [Association for Computing Machinery]
卷期号:: 16746-16752
标识
DOI:10.1145/3767308.3838666
摘要

Emotion recognition is a fundamental task in affective computing, and most current works consider multimodal fusion to integrate comprehensive information for better performance. Among the modalities, physiological signals present a specialty, as on the one side, they are hard to alter, providing more objective evidence for emotion understanding; on the other side, they require delicate equipment to be attached to the body, which is inconvenient and interfering for the monitored persons. Existing multimodal emotion research concerns more of non-physiological modalities, such as video, audio, and text, whereas physiological signals remain insufficiently explored. To bridge this gap, we introduce a new multimodal emotion dataset with remotely sensed physiological signals by millimeter-wave radar and facial videos. The new dataset is targeted to facilitate two research fields: 1) multimodal remote physiological signal measurement, and 2) physiological-behavioral fusion for emotion recognition with only remote sensors. Furthermore, beyond clip-level emotion annotations, we also provide fine-grained intensity labels, enabling more detailed temporal emotion analysis. Extensive benchmarks were established on the new dataset, including uni- and multimodal approaches for remote physiological signal measurement and emotion recognition. It is expected that the dataset and benchmarks provide a new testbed and open up new possibilities for future research. The dataset is available at: https://github.com/helloworld-YY/RemoPhysEmo.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
今后的应助被子岚采纳,获得10
刚刚
yxb发布了新的文献求助10
1秒前
1秒前
生动电脑完成签到 ,获得积分10
1秒前
2秒前
甜甜发布了新的文献求助20
2秒前
自觉的大凄完成签到 ,获得积分10
2秒前
Yyyyyyyyy完成签到,获得积分10
3秒前
晶晶完成签到,获得积分10
3秒前
研友_VZG7GZ的应助被SARON采纳,获得10
3秒前
3秒前
sujingbo完成签到 ,获得积分10
4秒前
哈哈发布了新的文献求助10
4秒前
123完成签到,获得积分10
5秒前
swkxwdh驳回了OK的应助
5秒前
6秒前
Hye_seon完成签到,获得积分10
6秒前
6秒前
彭于晏的应助被郇郇采纳,获得10
7秒前
zx完成签到,获得积分10
7秒前
沉默不评发布了新的文献求助10
7秒前
Layqiwook发布了新的文献求助10
7秒前
秋风的应助被科研通管家采纳,获得30
7秒前
7秒前
小冯同学完成签到,获得积分10
7秒前
小蘑菇的应助被科研通管家采纳,获得10
7秒前
温暖砖头发布了新的文献求助10
8秒前
orixero的应助被科研通管家采纳,获得10
8秒前
8秒前
小马甲的应助被科研通管家采纳,获得10
8秒前
NexusExplorer的应助被科研通管家采纳,获得10
8秒前
syvshc的应助被科研通管家采纳,获得10
8秒前
Ava的应助被科研通管家采纳,获得10
8秒前
852的应助被科研通管家采纳,获得10
8秒前
SciGPT的应助被科研通管家采纳,获得10
8秒前
852的应助被科研通管家采纳,获得10
8秒前
9秒前
xucc的应助被科研通管家采纳,获得10
9秒前
彭于晏的应助被科研通管家采纳,获得10
9秒前
rootree完成签到,获得积分10
9秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
Improving the wear resistance of vehicle gear oil by oil soluble LaF3 nanoparticles and typical additives of gear oil 500
Boundaries and Bodhisattvas 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8183241
求助须知:如何正确求助?哪些是违规求助? 9572493
关于积分的说明 21826218
捐赠科研通 7780115
什么是DOI,文献DOI怎么找? 3433982
关于科研通互助平台的介绍 2558103
邀请新用户注册赠送积分活动 2480192