Facial-Video-Based Physiological Signal Measurement: Recent advances and affective applications

Monitoring physiological changes [e.g., heart rate (HR), respiration, and HR variability (HRV)] is important for measuring human emotions. Physiological responses are more reliable and harder to alter compared to explicit behaviors (such as facial expressions and speech), but they require special contact sensors to obtain. Research in the last decade has shown that photoplethysmography (PPG) signals can be remotely measured (rPPG) from facial videos under ambient light, from which physiological changes can be extracted. This promising finding has attracted much interest from researchers, and the field of rPPG measurement has been growing fast. In this article, we review current progress on intelligent signal processing approaches for rPPG measurement, including earlier works on unsupervised approaches and recently proposed supervised models, benchmark data sets, and performance evaluation. We also review studies on rPPG-based affective applications and compare them with other affective computing modalities. We conclude this article by emphasizing the current main challenges and highlighting future directions.