rPPG-FuseNet: Non-contact heart rate estimation from facial video via RGB/MSR signal fusion

Remote photoplethysmography (rPPG) is an effective way of measuring Heart Rate (HR) from facial video of a person. Several methods have been proposed but measurement in realistic situation i.e, uncontrolled motion and illumination, is still a challenging task. It is observed that the performance of state-of-the-art methods often gets degraded in weekly illuminated environment. In this paper, a simple fusion based approach is proposed to reduce the performance degradation due to non-uniform illumination. An algorithm is proposed to fuse two complimentary color spaces i.e, Red, Green and Blue (RGB) and Multi Scale Retinex (MSR). The methodology includes salient spatiotemporal map generation from fusion of RGB and MSR. CNN is employed to map spatiotemporal features to corresponding HR value. To prove the efficacy of the proposed method, comparison is done with the state-of-the art methods with and without deep learning approach. Achieved experimental results proves that the proposed method yields comparable result for both inter and intra database testing on publicly available databases.