Tailoring the optical transfer function of nonlocal metasurfaces for targeted image processing via an automated inverse design framework

Nonlocal metasurfaces exhibit significant potential for advanced all-optical image processing by leveraging their exceptional capability to regulate spatial dispersion through precise tailoring of optical transfer functions (OTFs). However, the inverse design of specific OTFs remains challenging due to the inherently complex and highly nonlinear relationship between metasurface structural parameters and angular-dependent optical responses, which conventional empirical trial-and-error approaches struggle to address. To overcome this limitation, we propose an automated inverse design framework integrating a deep neural network acting as a forward predictor with Bayesian optimization. This framework enables automated OTF tailoring by optimizing metasurface structural parameters for targeted image processing operations at desired wavelengths within the 1,200-1,400 nm range. We validate the framework by designing nine dedicated silicon hollow brick metasurfaces: for each operational wavelength (1,250, 1,300, and 1,350 nm), three distinct devices are engineered to separately execute 2D second-order differentiation, 2D fourth-order differentiation, and 2D Gaussian high-pass filtering in transmission mode through targeted OTF engineering. These inversely designed nonlocal metasurfaces achieve a numerical aperture close to 0.4 and serve as fundamental components for edge detection and image sharpening. This intelligent, automated design paradigm dramatically accelerates the design process and significantly expands the scope of achievable functionalities for optical computing metasurfaces, paving the way for more sophisticated all-optical information processing systems.