Multi-modal multi-task deep neural networks for sleep disordered breathing assessment using cardiac and audio signals

BACKGROUND AND OBJECTIVE: Sleep disordered breathing (SDB) is one of the most common sleep disorders and has short-term consequences for daytime functioning while being a risk factor for several conditions, such as cardiovascular disease. Polysomnography, the current diagnostic gold standard, is expensive and has limited accessibility. Therefore, cost-effective and easily accessible methods for SDB detection are needed. Both cardiac and audio signals have received attention for SDB detection as they can be obtained with unobtrusive sensors, suitable for home applications. METHODS: This paper introduces a multi-modal multi-task deep learning approach for SDB assessment using a combination of cardiac and audio signals under the assumption that they can provide complementary information. We aimed to estimate the apnea-hypopnea index (AHI) and assess AHI-based SDB severity through the detection of SDB events, combined with total sleep time estimated from simultaneous sleep-wake classification. Inter-beat interval and electrocardiogram-derived respiration from the electrocardiogram, and Mel-scale frequency cepstral coefficients from concurrent audio recordings were used as inputs. We compared the performance of several models trained with different combinations of these inputs. RESULTS: Using cross-validation with a dataset comprising overnight recordings of 161 subjects, we achieved an F1 score of 0.588 for SDB event detection, a correlation coefficient of 0.825 for AHI estimation, and an accuracy of 57.8% for SDB severity classification (normal, mild, moderate, and severe). CONCLUSION: Results show that combining cardiac and audio signals can enhance the performance of SDB detection and highlight the potential of multi-modal data fusion for further research in this domain.