模式
模态(人机交互)
计算机科学
稳健性(进化)
推论
缺少数据
因果推理
人工智能
正交性
钥匙(锁)
机器学习
人机交互
数学
计算机安全
社会科学
生物化学
基因
社会学
计量经济学
化学
几何学
作者
Jae‐Hyuk Jang,Yooseung Wang,Changick Kim
出处
期刊:
日期:2024-03-18
卷期号:: 8070-8074
被引量:1
标识
DOI:10.1109/icassp48485.2024.10447257
摘要
Recently, multimodal prompting, which introduces learnable missing-aware prompts for all missing modality cases, has exhibited impressive performance. However, it encounters two critical issues: 1) The number of prompts grows exponentially as the number of modalities increases; and 2) It lacks robustness in scenarios with different missing modality settings between training and inference. In this paper, we propose a simple yet effective prompt design to address these challenges. Instead of using missing-aware prompts, we utilize prompts as modality-specific tokens, enabling them to capture the unique characteristics of each modality. Furthermore, our prompt design leverages orthogonality between prompts as a key element to learn distinct information across different modalities and promote diversity in the learned representations. Extensive experiments demonstrate that our prompt design enhances both performance and robustness while reducing the number of prompts.
科研通智能强力驱动
Strongly Powered by AbleSci AI