A Tutorial on Convex Design of Optimization Algorithms by Integral Quadratic Constraints

In this tutorial article, we expose the mechanisms underlying the design of optimization algorithms based on so-called dynamic integral quadratic constraints. These tools from robust control allow one to systematically construct accelerated first-order optimization algorithms with optimal guaranteed convergence rates by solving small-sized semidefinite programs. This is possible even if the information flow from and to the gradient is subject to nontrivial dynamics such as delays. Numerical experiments not only illustrate how to recover accelerated gradient algorithms by design but also unveil the flexibility of this approach gained from its embedding into systems theory and controller design, relying on the generalized plant framework.