On the use of differentiable optical models for lens and neural network co-design

Co-design methods started to incorporate neural networks a few years ago when deep learning showed promising results in computer vision. This requires the computation of the point spread function (PSF) of an optical system as well as its gradients with respect to the optical parameters so that they can be optimized using gradient descent. In previous works, several approaches have been proposed to obtain the PSF, most notably using paraxial optics, Fourier optics or differential ray tracers. All these models have limitations and strengths regarding their ability to compute a precise PSF and their computational cost. We propose to compare them in a simple co-design task to discuss their relevance. We will discuss the computational cost of these methods as well as their applicability.