Multimodal Deep Learning and Feature Fusion for Automated Sleep Apnea Event Detection Using ECG and Respiratory Signals

Abstract - Accurate and automated assessment of cardiorespiratory function is essential for physiological monitoring and the early detection of sleep-related disorders. Sleep apnea is a common but frequently underdiagnosed condition, affecting approximately 10% of middle-aged individuals, while polysomnography (PSG), the current diagnostic gold standard, is resource-intensive and requires substantial manual scoring. Advances in biosignal acquisition using modalities such as electrocardiography (ECG), photoplethysmography (PPG), electromyography (EMG), vibromyography (VMG), mechanomyography (MMG), and respiratory signals have enabled high-resolution and multimodal physiological analysis. In this context, deep learning techniques, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, and hybrid CNN–LSTM architectures, offer automated feature extraction and the ability to capture complex spatial and temporal patterns in physiological signals. This study investigates a multimodal deep-learning framework for automated sleep-apnea event detection by combining learned representations from raw ECG and respiratory signals with engineered cardiorespiratory features, including heart-rate-variability (HRV) and respiration-related descriptors. Using the MIT-BIH Polysomnographic Database, 10,274 non-overlapping 30-second windows from 18 records were analyzed. To prevent record-level data leakage, the records were divided into locked training, validation, and test sets. A logistic-regression model based on engineered features, a one-dimensional CNN based on raw physiological signals, and a CNN-based feature-fusion model were evaluated. The multimodal fusion model achieved a test ROC-AUC of 0.7341, PR-AUC of 0.5778, accuracy of 0.7317, precision of 0.5689, recall of 0.7204, specificity of 0.7372, and F1-score of 0.6358. Compared with logistic regression, the fusion approach improved ROC-AUC by 0.0222 and F1-score by 0.0665, while outperforming the raw-signal CNN by 0.0833 and 0.1440, respectively. These findings indicate that combining learned raw-signal representations with compact engineered cardiorespiratory descriptors can improve sleep-apnea event classification compared with either strategy alone. Future extensions could incorporate hybrid CNN–LSTM architectures for improved temporal modeling, Transformer-based approaches for multimodal signal fusion, and diffusion-based generative models for realistic biosignal augmentation and robustness evaluation. Nevertheless, substantial variability across held-out records highlights the need for external multi-center validation and patient-level statistical evaluation before clinical deployment.