Hierarchical Reinforcement Learning and Gain Scheduling-based Control of a Hypersonic Vehicle

A hierarchical reinforcement learning-based control strategy is introduced to facilitate state regulation for a hypersonic vehicle. To account for time-varying aerothermoelastic parameters in real-time, a hierarchical switching policy selects a subsystem from a larger set of potential subsystems. The selection depends on an approximation of the optimal value function of each subsystem. Integral concurrent learning is used to approximate the parametric uncertainties in each dynamical system. The approximate optimal control policy is proven to converge to a neighborhood of the optimal control policy. Uniformly ultimately bounded stability of each subsystem and stability of the overall switched system are proven using a Lyapunov-based stability analysis and dwell-time analysis.