A deep learning approach to estimate pulse rate by remote photoplethysmography

Abstract Objective. This study proposes a U-net shaped Deep Neural Network (DNN) model to extract remote photoplethysmography (rPPG) signals from skin color signals to estimate Pulse Rate (PR). Approach. Three input window sizes are used in the DNN: 256 samples (5.12 s), 512 samples (10.24 s), and 1024 (20.48 s). A data augmentation algorithm based on interpolation is also used here to artificially increase the number of training samples. Main results. The proposed model outperformed a prior-knowledge rPPG method by using input signals with window of 256 and 512 samples. Also, it was found that the data augmentation procedure only increased the performance for the window of 1024 samples. The trained model achieved a Mean Absolute Error (MAE) of 3.97 Beats per Minute (BPM) and Root Mean Squared Error (RMSE) of 6.47 BPM, for the 256 samples window, and MAE of 3.00 BPM and RMSE of 5.45 BPM for the window of 512 samples. On the other hand, the prior-knowledge rPPG method got a MAE of 8.04 BPM and RMSE of 16.63 BPM for the window of 256 samples, and MAE of 3.49 BPM and RMSE of 7.92 BPM for the window of 512 samples. For the longest window (1024 samples), the concordance of the predicted PRs from the DNNs and the true PRs was higher when applying the data augmentation procedure. Significance. These results demonstrate a big potential of this technique for PR estimation, showing that the DNN proposed here may generate reliable rPPG signals even with short window lengths (5.12 s and 10.24 s), suggesting that it needs less data for a faster rPPG measurement and PR estimation.