Homotopic policy iteration-based learning design for unknown linear continuous-time systems

Recent results have emerged that policy iteration is a powerful reinforcement learning tool in designing a stabilizing control policy for continuous-time systems with unknown system dynamics. Policy iteration involves a model-based initialization stage, i.e., seeking an initial stabilizing control policy, which is, however, dependent on the full system dynamics including the drift dynamics and system input matrix. To remove such model requirements, this paper utilizes a homotopy-based initialization strategy for policy iteration, wherein a stabilizing control policy for continuous-time systems is obtained by gradually moving a stable system to the original system. We propose two homotopic policy iteration-based stabilizing control schemes, namely, a model-based design and a model-free design using system data, which are proved to place unstable poles into a stable region. The effectiveness of the proposed designs is validated through an illustrative example.