Knowledge-Guided Reinforcement Learning with Artificial Potential Field-Based Demonstrations for Multi-Autonomous Underwater Vehicle Cooperative Hunting

Multi-AUV cooperative hunting requires autonomous underwater vehicles (AUVs) to strategize the encirclement of evaders while navigating around obstacles and other AUVs. Despite the promise of multi-agent reinforcement learning (MARL) in continuous control problems, its low sample efficiency poses a challenge in unknown environments and complex control scenarios. To overcome these limitations, we present a Knowledge-Guided Reinforcement Learning (KG-RL) approach, which integrates an Artificial Potential Field (APF) to enhance sample efficiency and operational safety. Our methodology is bifurcated into pre-training and fine-tuning phases. During the pre-training phase, an APF is employed to generate a concise set of demonstration trajectories that provide agents with foundational knowledge. Subsequently, the fine-tuning phase leverages real-time APF knowledge to direct the learning process, encouraging agents to balance following demonstrated actions with seeking out more optimal solutions. We assess the efficacy of our method through extensive simulations across diverse tasks, demonstrating its ability to expedite the learning process and yield more strategic decision-making. Our approach achieves superior results compared to traditional MARL benchmarks, particularly in learning efficiency, decision quality, and overall performance.