计算机科学
强化学习
人工智能
变压器
机器人
编码器
航路点
马尔可夫决策过程
凝视
计算机视觉
人机交互
机器学习
马尔可夫过程
实时计算
操作系统
数学
电压
量子力学
统计
物理
作者
Weiyuan Li,Ruoxin Hong,Jiwei Shen,Liang Yuan,Yue Lu
标识
DOI:10.1109/lra.2023.3241803
摘要
Substantial progress has been achieved in embodied visual navigation based on reinforcement learning (RL). These studies presume that the environment is stationary where all the obstacles are static. However, in real cluttered scenes, interactable objects (e.g. shoes and boxes) blocking the way of robots makes the environment non-stationary. Accordingly, the ego-centric visual agent will easily get stuck in the dilemma of finding the next waypoint as it struggles to decide whether to push the obstacles ahead. To handle the predicament, we formulate this interactive visual navigation as Partial Observed Markov Decision Process (POMDP). As the transformer encoder has demonstrated its superior ability to capture the spatial-temporal dependencies in natural language processing. We propose a transformer-based memory to empower the agents utilizing the historical interactive information. However, directly leveraging the transformer architecture in the RL settings is highly unstable. We further propose a surrogate objective to predict the next waypoint as the auxiliary task, which facilitates the representation learning and bootstraps the RL. We demonstrate our method in the iGibson environment and experimental results show a significant improvement over the interactive Gibson benchmark and the related recurrent RL policy both in the validation seen scenes and the test unseen scenes.
科研通智能强力驱动
Strongly Powered by AbleSci AI