计算机科学
对抗制
人工智能
机器人
控制(管理)
生成语法
机器人学
移动机器人
语言理解
计算机视觉
机器人控制
生成模型
自然语言
机器人学习
人机交互
弹道
语义学(计算机科学)
控制系统
机器人运动学
移动机器人导航
人机交互
社交机器人
遥控机器人学
语言模型
自然语言处理
反馈控制
控制工程
运动规划
作者
Ke Zhang,Zheng Fang,Enqi Zhao,Zicheng Sun,Jianwu Fang,Jie Huang,Eric Nichols,Randy Gómez,Bo He,Jianru Xue,Guangliang Li
标识
DOI:10.1109/tro.2026.3710412
摘要
Deep reinforcement learning (DRL) has achieved great success in many simulated and real-world robotic tasks. However, the difficulty of designing efficient and dense reward functions makes applying DRL to tackle complex long-horizon and open-world tasks a great challenge. Generative adversarial imitation learning (GAIL) can directly learn policies from the expert trajectories and generalize well in large and complex environments, but relies on high-quality demonstrations and can seldom surpass the performance of the demonstration. Recent work used additional human evaluative feedback to facilitate GAIL to learn faster and surpass the demonstrations, but still requires suboptimal demonstrations. Moreover, it is costly and difficult for human expert to provide relatively high-quality demonstrations and evaluative feedback for various tasks. To address the above issues, in this article, we propose generative adversarial self-imitation learning from demonstration and large language model (LLM) feedback (GASL$^{3}$MF), since LLMs encode rich commonsense knowledge and can perform a variety of reasoning tasks. GASL$^{3}$MF allows a robot to learn from poor demonstrations and gradually replace them with its own good trajectories evaluated by LLM feedback. Our results in four physics-based control tasks and a mobile robot navigation task show that, even with demonstrations of poor performance or not completing the task, GASL$^{3}$MF can learn faster with close to optimal performance, and generalize well to different environments and the real world with sim-to-real adaptation. Further analysis shows that the overall distribution of LLM feedback closely resembles that of human feedback and remains closer to that of ground-truth rewards than human feedback. Finally, our GASL$^{3}$MF method works regardless of the LLM employed, and the LLM feedback from different LLMs remain robust across tasks and even better consistency than human feedback for robot learning in some tasks. These results shed light on the potential of robot imitation learning from even poor or failed demonstrations and broaden its application to a wide range of real-world tasks.
科研通智能强力驱动
Strongly Powered by AbleSci AI