Robust Adaptive Dynamic Programming Control for Uncertain Discrete-Time Nonlinear Systems

This article studies two robust adaptive dynamic programming (ADP) approaches for uncertain discrete-time (DT) nonlinear systems. Since the uncertainty is implicit in the traditional Hamilton-Jacobi–Bellman (HJB) equation, it is difficult to deal with the uncertainty. In this article, the Taylor series approximation technique is utilized to convert the traditional HJB equation into an explicit form of the uncertainty. In virtue of the first-order Taylor series approximation technique, a robust first-order approximate HJB equation is established. To further improve the approximation accuracy, a robust second-order approximate HJB equation is exploited by using the Hessian matrix of the value function. It is shown that the second-order approximate HJB equation could be extended to the uncertain DT linear systems. Aiming at obtaining the solutions of the two robust approximate HJB equations, we propose two corresponding policy iteration (PI) algorithms. More importantly, the convergence and optimality of the designed PI algorithms are clarified. Finally, a numerical case is conducted to test the validity of the designed robust DT PI ADP approaches.