LLM Coach: Reward Shaping for Reinforcement Learning-Based Navigation Agent

Despite the widespread deployment of Large Language Models (LLMs) in embodied intelligent agents for their promising human-like reasoning capabilities, they remain hindered by local infeasibility and unpredictable cloud latency. To address this challenge, we propose LLM-Coach, a reinforcement learning framework that utilizes LLM-driven reward shaping for navigation on water surface during the training phase. Our approach enables models to integrate the common-sense knowledge embedded in LLMs while eliminating runtime dependency on these resource-intensive models. We implement the proposed framework in the European Ship Simulator environment, introducing two specialized LLM agents: an Observation Agent for real-time state extraction and an Evaluation Agent for policy assessment. Through synergistic collaboration, these agents facilitate the training of a deep neural network-based navigation model (ESSNet) that operates efficiently during execution. Finally, experimental results demonstrate that our approach outperforms traditional reinforcement learning methods while ensuring operational robustness.