A smart embedded imaging system for true color face analysis

One of the most difficult challenges in counter-terrorism, crime fighting and surveillance mission is to accurately identify people from an image/video footage to catch the shortlisted terrorists and criminals. For this purpose, imaging devices used in the video surveillance systems are being improved in many aspects such as spatial resolution, frame rate, dynamic range, spectral characteristics in order to achieve better imaging performance for both monitoring and automatic detection/recognition tasks. These development efforts aim to improve the basic imaging characteristics of the device, such as the ratio of average color values (i.e red(R)/green(B) and blue(B)/green(G)) of the video footage, irrespective of high-level semantic knowledge such as presence and locations of the monitored objects/individuals in the scene by the camera. Nevertheless, a scene under multiple lighting conditions with different spectral characteristics cannot be accurately viewed with a standard imaging system in terms of color distribution. On the other hand, color is one of the most crucial factors regarding the objects/individuals identification for both operators and AI-based systems, and capturing true color features under various harsh environments is the most critical issue. In this study, a cognitive imaging system prototype is proposed to capture the true color distribution over the human faces by taking into account the locations of the detected faces with a developed smart camera. This is achieved by developing a smart Auto-White Balance (AWB) algorithm which use only the region of interest (ROI) to compute/measure the color ratios of the video frame. The current ROI is selected by intelligently and sequentially traversing among the detected faces to effectively handle whole faces in the scene. The experimental results show that the proposed "cognitive" camera can achieve impressive improvement over the color accuracy/constancy for the detected human faces under the empirically illuminated conditions.