Some Comments on Preconditioning for Normal Equations and Least Squares

The solution of systems of linear(ized) equations lies at the heart of many problems in scientific computing. In particular, for large systems, iterative methods are a primary approach. For many symmetric (or self-adjoint) systems, there are effective solution methods based on the conjugate gradient method (for definite problems) or MINRES (for indefinite problems) in combination with an appropriate preconditioner, which is required in almost all cases. For nonsymmetric systems there are two principal lines of attack: the use of a nonsymmetric iterative method such as GMRES or transformation into a symmetric problem via the normal equations and application of LSQR. In either case, an appropriate preconditioner is generally required. We consider the possibilities here, particularly the idea of preconditioning the normal equations via approximations to the original nonsymmetric matrix. We highlight dangers that readily arise in this approach. Our comments also apply in the context of linear least squares problems.