Illusion of convergence: Search space geometry in radiotherapy treatment plan optimization

Abstract Background Treatment plan optimization is foundational for radiotherapy. However, the geometry of the underlying search space where feasible solutions reside remains poorly characterized. Understanding this search space is crucial as it: (1) Exposes hidden limitations in objective functions; (2) Reveals the nature of convergence; and (3) Informs the choice of optimization algorithms and initialization strategies. Purposes To characterize the geometry of the search space, examine the nature of convergence, and provide theoretical guidance for empirical rules in clinical practice. Materials and Methods This study examines fluence map optimization (FMO) for intensity‐modulated radiotherapy (IMRT) using a L‐ BFGS based framework. Hessian matrix of objective function was derived, and its eigenvalues were determined by the voxels actively contributing to the objective and the singular value decomposition (SVD) of the dose‐deposition matrix. Numerical analysis includes estimating dominant eigenvalues and corresponding eigenvectors. Perturbation analysis was conducted along individual beamlet intensity and principal eigenvectors for visualization. The primary focus was on quadratic dose–volume objectives (DVOs), with extensions to generalized equivalent uniform dose (gEUD). Results Eigenvalue spectra showed highly anisotropic curvature near the optimum, dominated by broad flat regions with minimal sensitivity in most directions. Perturbation plots revealed that many beamlets were either insensitive or exhibited step‐like, discontinuous behavior. Convergence often occurred in such flat regions, producing an “illusion of convergence” that did not guarantee the lowest basin. Optimization track comparisons showed that second‐order methods, by incorporating curvature, advanced more efficiently than first‐order methods. Conclusion For DVO‐based FMO, search space is dominated by flat plateaus at different altitudes, making convergence sensitive to both optimizer and initialization. The findings explain why clinical plans are often locally but not globally optimal, while still acceptable in quality. The analysis provides a theoretical foundation for longstanding empirical practices and offers guidance on optimizer selection and initialization strategies in treatment plan optimization.