Numerical stability and efficiency in Zernike fitting for non-standard circular apertures

Retrieving standard circular Zernike coefficients from wavefronts measured over non-standard circular apertures is crucial for modern optical engineering. While direct least-squares fitting and the orthogonal transformation method are commonly used, a systematic comparison of their numerical stability, computational costs, and noise robustness remains lacking. In this paper, we present a comparative study of five established algorithms [least-squares, Gram–Schmidt, Cholesky factorization, QR factorization, and singular value decomposition (SVD)], introduce two short-recurrence variants to investigate the computational speed limits of recursive orthogonalization, and rigorously prove their mathematical equivalence under exact arithmetic and characterize how floating-point errors cause this equivalence to break down differently across methods. However, simulations on annular and eccentric apertures reveal that although all methods are reliable under well-conditioned masks, least-squares and Cholesky suffer precision loss and numerical collapse under ill-conditioned apertures. Furthermore, non-regularized methods suffer from noise amplification, making truncated SVD the most reliable method among those evaluated for retrieving physically meaningful coefficients. Based on these findings, we provide a practical guide for algorithm selection in optical engineering, helping researchers choose appropriate methods based on their specific measurement conditions.