Unconstrained Multivariate Gradient-Based Minimization

Unconstrained multivariate gradient-based minimization is introduced by means of search direction-producing methods, focusing on steepest descent and Newton's method. Issues with both methods are discussed, highlighting what happens in the case of locally nonconvex functions, particularly in Newton's method. Linesearch is introduced, effectively rendering multidimensional optimization into a sequence of one-dimensional searches along the ray of the search directions produced. Linesearch criteria are discussed, such as the Armijo first condition, and efficient ways to cut the step size are discussed.