Robust Prediction of Cardiorespiratory Signals from a Multimodal Physiological System on the Upper Arm

Many commercial wearable sensor systems typically rely on a single continuous cardiorespiratory sensing modality, photoplethysmography (PPG), which suffers from inherent biases (i.e., differences in skin tone) and noise (e.g., motion and pressure artifacts). In this research, we present a wearable device that provides robust estimates of cardiorespiratory variables by combining three physiological signals from the upper arm: multiwavelength PPG, single-sided electrocardiography (SS-ECG), and bioimpedance plethysmography (BioZ), along with an inertial measurement unit (IMU) providing 3-axis accelerometry and gyroscope information. We evaluated the multimodal device on 16 subjects by its ability to estimate heart rate (HR) and breathing rate (BR) in the presence of various static and dynamic noise sources (e.g., skin tone and motion). We proposed a hierarchical approach that considers the subject's skin tone and signal quality to select the optimal sensing modality for estimating HR and BR. Our results indicate that, when estimating HR, there is a trade-off between accuracy and robustness, with SS-ECG providing the highest accuracy (low mean absolute error; MAE) but low reliability (higher rates of sensor failure), and PPG/BioZ having lower accuracy but higher reliability. When estimating BR, we find that fusing estimates from multiple modalities via ensemble bagged tree regression outperforms single-modality estimates. These results indicate that multimodal approaches to cardiorespiratory monitoring can overcome the accuracy-robustness trade-off that occurs when using single-modality approaches.