A 120 Frames/s CMOS Image Sensor With 8.19 TOPS/W Computing-In-Pixel for Energy-Efficient Low-Latency Face Detection

We present a CMOS image sensor (CIS) with on-chip energy-efficient face detection circuits for low-latency vision-based tracking systems. The embedded charge-domain computing circuits compute early stages of cascaded machine-learning classifiers for the images acquired from a global-shutter pixel array while simultaneously providing the digitized image readout. This approach improves the system energy–latency product by $2.3{\times }$ , compared to the conventional high-speed tracking systems, because the backend computational load can be significantly reduced with no latency overhead. In addition, charge-domain computing within the pixel array allows for little energy overhead. The prototype image sensor was fabricated in a 180-nm CIS process with $240{\times }240$ voltage-domain global shutter pixels. On-chip face detection rejects 98% of 3500 scanning windows and reduces the backend workload by $5.3{\times }$ , while consuming only 421 pJ/pixel at 120 frames/s. This translates to the best in-class energy efficiency of 8.2 TOPS/W among CIS with near-sensor computing. In addition, the fabricated sensor supports illumination-invariant and multi-scale detection for robust operation, which is important in real applications.