Cambrian-S: Towards Spatial Supersensing in Video

作者
Shusheng Yang,Jihan Yang,Peiyan Huang,Ellis Brown,Zihao Yang,Xingxing Zhu,Shengbang Tong,Zhongqiang Zheng,Yifan Xu,Minghua Wang,D. Lu,Rob Fergus,Yann LeCun,Feifei Li,Saining Xie
出处
期刊:Cornell University - arXiv [Cornell University]
标识
DOI:10.48550/arxiv.2511.04670
摘要

We argue that progress in true multimodal intelligence calls for a shift from reactive, task-driven systems and brute-force long context towards a broader paradigm of supersensing. We frame spatial supersensing as four stages beyond linguistic-only understanding: semantic perception (naming what is seen), streaming event cognition (maintaining memory across continuous experiences), implicit 3D spatial cognition (inferring the world behind pixels), and predictive world modeling (creating internal models that filter and organize information). Current benchmarks largely test only the early stages, offering narrow coverage of spatial cognition and rarely challenging models in ways that require true world modeling. To drive progress in spatial supersensing, we present VSI-SUPER, a two-part benchmark: VSR (long-horizon visual spatial recall) and VSC (continual visual spatial counting). These tasks require arbitrarily long video inputs yet are resistant to brute-force context expansion. We then test data scaling limits by curating VSI-590K and training Cambrian-S, achieving +30% absolute improvement on VSI-Bench without sacrificing general capabilities. Yet performance on VSI-SUPER remains limited, indicating that scale alone is insufficient for spatial supersensing. We propose predictive sensing as a path forward, presenting a proof-of-concept in which a self-supervised next-latent-frame predictor leverages surprise (prediction error) to drive memory and event segmentation. On VSI-SUPER, this approach substantially outperforms leading proprietary baselines, showing that spatial supersensing requires models that not only see but also anticipate, select, and organize experience.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刚刚
易安发布了新的文献求助10
刚刚
dsfsd发布了新的文献求助10
刚刚
Pzuzu发布了新的文献求助10
刚刚
刚刚
ljp完成签到,获得积分10
刚刚
刚刚
suxiang完成签到,获得积分10
刚刚
微笑谷雪应助温暖的定格采纳,获得10
1秒前
打打应助ivar采纳,获得10
1秒前
1秒前
BBbang440发布了新的文献求助10
1秒前
灰灰鹤发布了新的文献求助10
1秒前
杜拉拉发布了新的文献求助20
2秒前
2秒前
蓝天应助自然的霸采纳,获得10
3秒前
柒_l完成签到 ,获得积分10
3秒前
微微完成签到,获得积分10
3秒前
3秒前
3秒前
3秒前
高大如冬发布了新的文献求助30
4秒前
march完成签到 ,获得积分10
4秒前
4秒前
111发布了新的文献求助10
5秒前
彭彭发布了新的文献求助10
5秒前
5秒前
mAnda完成签到,获得积分10
5秒前
6秒前
6秒前
赘婿应助missinglotta采纳,获得10
6秒前
自由大碗发布了新的文献求助10
6秒前
章半仙完成签到,获得积分10
7秒前
7秒前
独特啤酒发布了新的文献求助10
7秒前
Nico_Ding完成签到,获得积分10
7秒前
爱撒娇的怜珊完成签到,获得积分10
8秒前
轱辘牛发布了新的文献求助20
8秒前
8秒前
wu发布了新的文献求助10
8秒前
高分求助中
Principles of Economics, 11th Edition 10000
University Physics with Modern Physics, 16th edition 10000
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
Arthritis and Related Conditions, An Issue of Orthopedic Clinics 1000
Development of a Bridge Weigh-In-Motion System: A technology to convert the bridge response to the passage of traffic into data on vehicle configurations, speeds, times of travel and weights 1000
ズームレンズの光学設計に関する研究 800
Fundamentals of Pharmaceutical and Biologics Regulations: A Global Perspective, Second Edition 700
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 工程类 有机化学 化学工程 生物化学 计算机科学 内科学 物理 复合材料 催化作用 细胞生物学 无机化学 光电子学 物理化学 电极 基因
热门帖子
关注 科研通微信公众号,转发送积分 7292168
求助须知:如何正确求助?哪些是违规求助? 8911140
关于积分的说明 18863722
捐赠科研通 6959278
什么是DOI,文献DOI怎么找? 3209566
关于科研通互助平台的介绍 2379066
邀请新用户注册赠送积分活动 2185369