Bridging vision-language reasoning and learned control for legged navigation in the wild

计算机科学 人工智能 强化学习 机器人 地形 桥接(联网) 软件部署 组分(热力学) 运动规划 空间智能 自动化 人机交互 移动机器人 移动机器人导航 控制(管理) 分割 敏捷软件开发 机器人学 导航系统 位于 计算机视觉 语义学(计算机科学) 自主系统(数学) 口译(哲学) 可靠性(半导体) 运动(物理)
作者
Takahiro Miki,Robin Schmid,Manthan Patel,Turcan Tuna,Julia Richter,Julian Nubert,Joonho Lee,Matthias F. Mueller,Marco Hutter
出处
期刊:Science robotics [American Association for the Advancement of Science]
卷期号:11 (120)
标识
DOI:10.1126/scirobotics.aee1549
摘要

Autonomous navigation in diverse real-world environments remains one of the greatest challenges in robotics. It demands both robust low-level motion control to handle diverse terrain and high-level reasoning to interpret environmental cues. We present ROAM (Robust Open-world Autonomous Mobility), a system that enables legged robots to perform kilometer-scale navigation in complex alpine and urban settings. ROAM combines the world knowledge of a vision-language model (VLM), pretrained on internet-scale data, with robust locomotion and local planning policies learned in simulation via reinforcement learning (RL). The VLM provides common-sense reasoning and semantic interpretation of the scene, assessing risks, interpreting signs, and producing semantic labels with associated navigation costs. An open-vocabulary segmentation module converts these labels into a dense, real-time risk map, which conditions two pretrained RL policies responsible for agile locomotion and local navigation: effectively coupling high-level VLM reasoning with low-level physical control. Real-world experiments showed the system’s ability to autonomously navigate a wide range of complex real-world environments. The robot completed kilometer-long hikes with minimal intervention, interpreted signs to avoid danger, and evaluated its environment to identify safe areas on which to step. This work represents a step toward the reliable deployment of systems in unstructured environments. However, it also underscores the challenges that remain in VLM reliability and spatial reasoning, indicating that foundational models are a crucial component but not yet a complete solution for navigation in the wild.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
3秒前
4秒前
6秒前
青松完成签到,获得积分10
6秒前
共享精神的应助被缓慢念波采纳,获得10
8秒前
8秒前
美好明辉发布了新的文献求助10
8秒前
烫水一壶完成签到,获得积分10
8秒前
9秒前
Ying发布了新的文献求助10
9秒前
Akim的应助被darkegg采纳,获得10
9秒前
牧百川发布了新的文献求助10
10秒前
沉沉叠叠发布了新的文献求助10
10秒前
Jun的应助被晴天采纳,获得10
10秒前
gengar完成签到,获得积分20
11秒前
11秒前
酷波er的应助被禾十采纳,获得10
11秒前
冷艳的河马完成签到,获得积分10
13秒前
14秒前
14秒前
SPP完成签到 ,获得积分10
15秒前
15秒前
Dispute完成签到,获得积分10
15秒前
樊家圣完成签到 ,获得积分10
15秒前
小呆鹿完成签到,获得积分10
16秒前
斯文败类的应助被热心的柚子采纳,获得10
17秒前
所所的应助被bailingli采纳,获得10
18秒前
18秒前
19秒前
深情安青的应助被xiaot采纳,获得10
19秒前
19秒前
19秒前
20秒前
领导范儿的应助被美好明辉采纳,获得10
20秒前
科研通AI6.3的应助被牧百川采纳,获得10
21秒前
21秒前
21秒前
21秒前
Jason完成签到 ,获得积分10
22秒前
贪玩的醉柳完成签到,获得积分10
22秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
Improving the wear resistance of vehicle gear oil by oil soluble LaF3 nanoparticles and typical additives of gear oil 500
Boundaries and Bodhisattvas 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8183621
求助须知:如何正确求助?哪些是违规求助? 9572918
关于积分的说明 21828010
捐赠科研通 7780809
什么是DOI,文献DOI怎么找? 3434031
关于科研通互助平台的介绍 2558243
邀请新用户注册赠送积分活动 2480496