Reinforcement Learning‐Based Model Predictive Path Integral Control for Obstacle Avoidance of Autonomous Underwater Vehicles

ABSTRACT Autonomous underwater vehicles (AUVs) face substantial challenges in obstacle avoidance due to the complex, dynamic nature of underwater environments and inherent sensing limitations. This study introduces a novel optimization framework that addresses these challenges by synergistically integrating advanced sampling strategies with reinforcement learning (RL) and model predictive path integral (MPPI) algorithms. The proposed framework strategically leverages the complementary strengths of both approaches: MPPI's proficiency in short‐term trajectory prediction combined with RL's exploratory capabilities and end‐to‐end training paradigm. This integration enables AUVs to rapidly adapt to environmental perturbations, make efficient real‐time obstacle avoidance decisions, continuously adjust to increasingly complex underwater scenarios, and achieve long‐term safe navigation objectives. To evaluate the efficacy of this RL‐MPPI hybrid approach, comprehensive numerical simulations were conducted across diverse underwater environmental conditions, encompassing both static and dynamic obstacles. The simulation results demonstrate enhanced adaptability and responsiveness in complex underwater environments, improved predictive accuracy and stability in obstacle avoidance maneuvers, and effective navigation through static and dynamic underwater scenarios while maintaining robust predictive characteristics. Quantitatively, the proposed method reduces the average cost value by 9.3% and average execution time by 2.9% compared with traditional MPPI in water‐free environments. Furthermore, in the presence of unknown water flow, it achieves a 7.2% reduction in average cost value and a 1.6% decrease in average execution time. This study contributes to the advancement of underwater robotics by offering a robust, adaptive, and computationally efficient approach to collision prevention for AUVs. The proposed framework demonstrates considerable promise for enhancing AUV capabilities in safe and efficient navigation through increasingly challenging underwater environments.