Horizon-Greedy Q -Ensembles Regularized Decision Transformer: An Offline Reinforcement Learning Approach for Robotic Tasks

Offline reinforcement learning (RL) offers a promising paradigm for learning policies from precollected datasets. Nonetheless, applying it to robotic control poses significant challenges, including non-Markovian dynamics and the scarcity of high-quality demonstrations, both of which can undermine the performance of existing methods. To address these issues, this work introduces the horizon-Greedy Q-ensembles Regularized decision Transformer (GQRT), an offline RL algorithm tailored for robotic tasks. GQRT leverages a transformer-based policy for trajectory modeling, thereby enabling effective long-horizon decision-making in non-Markovian settings. To alleviate the lack of expert demonstrations, we develop a multistep horizon-greedy policy evaluation mechanism that stitches suboptimal sequences into improved trajectories. Furthermore, to cope with mixed-quality demonstrations collected from diverse sources, we incorporate a Q-ensemble with lower confidence bound regularization, which ensures more stable and reliable value estimation. Extensive experiments on robotic locomotion and manipulation benchmarks demonstrate that GQRT achieves state-of-the-art performance, validating its robustness in complex robotic scenarios.