Prioritized experience replay based reinforcement learning for adaptive tracking control of autonomous underwater vehicle

A novel adaptive trajectory tracking control method is proposed in this paper for autonomous underwater vehicle systems with external disturbances. The control action is composed of sliding mode control and action-depended heuristic dynamic programming (ADHDP) controller to track the expected sailing position and angle in the horizontal coordinate system with fixed depth. As an auxiliary control of sliding mode control, the ADHDP controller observes the difference between the actual sailing position/angle and the expected sailing position/angle, and adaptively provides corresponding supplementary control actions using a data-driven fashion. At the same time, we design a weight-related priority experience replay (PER) technology to update the online weight network by using the relevant historical data stored in the database to improve the learning rate. The proposed algorithm can adjust parameters online under various conditions. Furthermore, it is very suitable for underwater vehicle systems with parameter uncertainties and external disturbances. Based on Lyapunov stability method, the stability of closed-loop system state and network weight error is analyzed. Finally, the validity of our control strategy is verified by simulation.