Real-time Obstacle Avoidance for AUV Based on Reinforcement Learning and Dynamic Window Approach

As an important tool for exploring the ocean, autonomous underwater vehicle (AUV) plays an irreplaceable role in various marine activities. Due to the complexity and uncertainty of the marine environment, AUV is required to develop in a more intelligent direction. How to ensure that AUV avoids obstacles and reaches the target point smoothly is a key research issue of the AUV. The dynamic window approach (DWA) is adopted to AUV in this paper to achieve AUV's autonomous obstacle avoidance for static obstacles. The DWA is used to search for the optimal velocity command in its admissible velocity space by maximizing the objective function, however, the weights of its objective function are constant, which makes AUV lack flexibility in complex environments, and even unable to avoid obstacles. To address the above problem, reinforcement learning is introduced to optimize DWA. Q-learning, a reinforcement learning algorithm, is used to learn the weights of the DWA's objective function, which enables appropriate weights can be selected in different environments and improves the applicability of DWA in the complex environment. Compared with the original DWA, the DWA combined with Q-learning is effective and suitable for complex obstacle environments.