Stability and ψ-algebraic decay of the solution to ψ-fractional differential system

Abstract In this article, we focus on stability and ψ -algebraic decay (algebraic decay in the sense of ψ -function) of the equilibrium to the nonlinear ψ -fractional ordinary differential system. Before studying the nonlinear case, we show the stability and decay for linear system in more detail. Then we establish the linearization theorem for the nonlinear system near the equilibrium and further determine the stability and decay rate of the equilibrium. Such discussions include two cases, one with ψ -Caputo fractional derivative, another with ψ -Riemann–Liouville derivative, where the latter is a bit more complex than the former. Besides, the integral transforms are also provided for future studies.