凤凰
计算机科学
动作(物理)
运动(物理)
反射(计算机编程)
自我反省
人工智能
物理
地理
心理学
量子力学
程序设计语言
精神分析
大都市区
考古
作者
Wenke Xia,Ruoxuan Feng,Dong Wang,Di Hu
出处
期刊:Proceedings
[Institute of Electrical and Electronics Engineers]
日期:2025-06-10
卷期号:: 6981-6990
被引量:24
标识
DOI:10.1109/cvpr52734.2025.00655
摘要
Building a generalizable self-correction system is crucial for robots to recover from failures. Despite advancements in Multimodal Large Language Models (MLLMs) that empower robots with semantic reflection ability for failure, translating semantic reflection into "how to correct" fine-grained robotic actions remains a significant challenge. To address this gap, we build the Phoenix framework, which leverages motion instruction as a bridge to connect high-level semantic reflection with low-level robotic action correction. In this motion-based self-reflection framework, we start with a dual-process motion adjustment mechanism with MLLMs to translate the semantic reflection into coarse-grained motion instruction adjustment. To leverage this motion instruction for guiding "how to correct" fine-grained robotic actions, a multi-task motion-conditioned diffusion policy is proposed to integrate visual observations for high-frequency robotic action correction. By combining these two models, we could shift the demand for generalization capability from the low-level manipulation policy to the MLLMs-Driven motion adjustment model and facilitate precise, fine-grained robotic action correction. Utilizing this framework, we further develop a lifelong learning method to automatically improve the model’s capability from interactions with dynamic environments. The experiments conducted in both the RoboMimic simulation and real-world scenarios prove the superior generalization and robustness of our framework across a variety of manipulation tasks. Our code is released at https://github.com/GeWu-Lab/Motion-Based-Self-Reflection-Framework.
科研通智能强力驱动
Strongly Powered by AbleSci AI