Memory-Efficient and Real-Time SPAD-based dToF Depth Sensor with Spatial and Statistical Correlation

Single Photon Avalanche Diode (SPAD)-based direct time-of-flight (dToF) depth sensors are widely used in Internet of Things (IoT) devices due to their high accuracy. Existing SPAD-based dToF sensors measure depth by continually accumulating the depth-measured value in a histogram. However, histogram-based methods typically have low convergence speed (~10 frames per second (FPS)) and large memory overhead (MB-level), hindering their use in real-time embedded IoT devices. To overcome these two challenges, we propose SSC, a histogram-free Spatial and Statistical Correlation based depth measurement method. On the one hand, SSC applies the spatial correlation of the adjacent pixels to accelerate the convergence speed. On the other hand, SSC explores the statistical correlation of depth measurements to reduce the memory overhead. In order to implement SSC with small hardware area and low power, we design mert-dToF, a memory-efficient and real-time dToF sensor for efficient execution. mert-dToF abstracts mainly operations in SSC into four basic operators and designs corresponding hardware with a fine-grained pipeline to maximize resource reuse and computational parallelism. Extensive experiments show that compared with state-of-the-art (SOTA) histogram-based dToF sensors, mert-dToF achieves ~8% accuracy improvement and 7.80× speedup (from 6.24 FPS to 48.70 FPS). The memory overhead is reduced by up to 60.91% (from 48 KB to 18.75 KB).