Ultra-fast exposure enhanced imaging with SPAD arrays based on super-resolution deep learning

<sec>In recent years, with wide spread applications of high-sensitivity single-photon detectors, especially in the fields of quantum imaging and optical imaging, many important achievements have been made. and micro light imaging technology based on single-photon level has gradually become an important branch of high-resolution imaging systems. At present, the main single-photon detectors are single-photon counting avalanche diode (SPAD) sensors and support pixel arrays of different sizes, ranging from single-pixel detector sizes to tens of thousands of pixel SPAD arrays. The process structure of single-pixel SPAD detectors is relatively simple, and they are often used as the first choice for low-light imaging due to their high sensitivity, small size, and low cost. However, due to the lack of spatial resolution, single-pixel SPADs can only detect signals at a single location and cannot provide spatial information, and they are usually used in conjunction with a spatial light modulator DMD or SLM with spatial resolution to reconstruct two-dimensional (2D) images through compressed sensing or quantum correlation. Although single-pixel detectors can provide ns-level or even ps-level temporal resolution, they are limited by the frame rate of the spatial light modulator (SLM). For example, the fastest digital micromirror device (DMD) is a type of SLM with a frame rate of 22 kHz, which means that the imaging rate of a single-pixel camera is typically limited to seconds, and this camera typically uses an SPAD and an SLM for single-photon imaging tasks. This makes it a challenge to significantly improve the imaging speed, especially when higher imaging resolution is required, such as those exceeding hundreds of thousands of pixels. Assuming that the imaged object is a fast-moving dynamic target, a few seconds of imaging rate will inevitably lead to dynamic blurring, which also poses a challenge to the fast real-time performance of single-photon imaging systems.</sec><sec>The SPAD array sensor retains the excellent sensitivity, low dark count rate, and high temporal resolution of single-pixel SPAD sensors. Due to the improvement of the fabrication process, multiple sensors and readout circuits are fabricated on the same chip, thus leading to the development of spatially resolved SPAD array camera. However, the integrated design of SPAD arrays with multiple pixels and circuits inevitably leads to cross-crosstalk between pixels. This crosstalk can significantly affect the accuracy of the signal. Additionally, the fill factor of such array cameras is typically low. Although the fill factor can be improved by methods such as three-dimensional (3D) stacking and microlens arrays, the spatial utilization is still to be improved in comparison with single-pixel SPADs. However, it is undeniable that SPAD arrays perform well in high dynamic range photon flux detection and high frame rate photon counting measurements due to the parallel processing of multiple detectors. Currently, commercial SPAD arrays integrate hundreds of thousands of detector pixel units, thereby providing excellent spatial resolution. Unfortunately, due to manufacturing processes and various challenges, the SPAD array cameras have been used in high-quantification bit deep sampling mode to acquire high-resolution single-photon intensity imaging. Its exposure time is limited to milliseconds. It is difficult to avoid dynamic blurring during the imaging exposure time when the motion frequency of the dynamic target reaches kHz or higher. Although the quantification bit depth can be sacrificed to shorten the minimum exposure time of array camera to the ns level, too short an exposure time can result in the SPAD array capturing the sparse photon data contaminated by a large amount of noise. Therefore, reliable photon denoising methods need to be developed. These methods are essential for effectively separating background noise from the actual signals, thereby improving the signal-to-noise ratio of the imaging system. Therefore, the real-time performance of the imaging system at the expense of quantification sampling accuracy still needs to be further optimized.</sec><sec>In order to solve the problem of limited imaging quality and rate of SPAD arrays under very short exposure times, we propose a single-photon imaging enhanced deep neural network combined with super-resolution deep learning in this work. By constructing a single-photon image dataset with dynamic exposure times and conducting adaptive training, high-fidelity reconstruction of low signal-to-noise ratio single-photon images can be achieved under ultra-short exposure time. In the experiments, the enhanced reconstruction of low-quality fan images (PSNR/SSIM, 6.54 dB/0.18) under very low-light conditions is achieved, with an exposure time of only 1 μs and an average photon number of less than 0.5 photons (PNSR/SSIM, 13.21 dB/0.34). And the images are effectively improved by +7.21 dB/+0.16 for PSNR and SSIM. The passive remote enhanced reconstruction is performed on the drone at a distance of 5.19 km, with an imaging exposure time of 5 μs, and an effective PSNR and SSIM enhancement of +4.78 dB/+0.2. This method provides a new technical solution for SPAD arrays for achieving ultra-fast-exposure high-quality imaging.</sec>