Reinforcement Learning Applied to Hexapod Robot Locomotion: An Overview

Six-legged robots are very useful in environments with obstacles of a size comparable to its own. However, the locomotion problem of hexapod robots is complex to solve due to the number of degrees of freedom and unknown environments. Nevertheless, the problem definition of Reinforcement Learning fits naturally for solving the robot locomotion problem. Reinforcement Learning has acquired great relevance in the last decade since it has achieved human-level control for specific tasks. This article presents an overview of Reinforcement Learning methods that have been successfully applied to the six-legged robot locomotion problem. First, a description and some achievements of reinforcement learning will be introduced, followed by examples of hexapod robots throughout history focusing on their locomotion systems. Secondly, the locomotion problem for a six-legged hexapod robot will be defined, with special attention to both, the gait and leg motion planning. Thirdly, the classical framework of reinforcement learning will be introduced and the Q-learning algorithm, which is one of the most used Reinforcement Learning algorithms in this context, will be revised. Finally, reinforcement learning methods applied to six-legged robot locomotion will be extensively discussed followed by open questions.