Are multimodal AIs predicting proxies for age and sex in sleep-based cardiovascular risk models?

Several recent works have focused on predicting risk for incident cardiovascular disease (CVD) from routine sleep studies using AI. These works aim to utilize the breadth of multimodal sleep study signal data to better identify and stratify people who are more at risk for disease and specific groups that would benefit from therapy. The predominant sleep study cohorts used for CVD risk predictive modeling are the Sleep Heart Health Study (SHHS), the Multi-Ethnic Study of Atherosclerosis (MESA), and the Osteoporotic Fractures in Men Study (MrOS). In this perspective, we show that these datasets have primarily older participants, with clinically diagnosable sleep apnea based on the apnea hypopnea index. Further, we show that two sets of simple Cox regression models, an age-sex Cox model and a traditional risk factors Cox model based on the American Heart Association's PREVENT risk score, can predict six different CVD outcomes and closely match results of AI predictive models, including SleepFM. Ultimately, our results suggest that sleep AI models may be over-relying on or learning proxies for demographic features when estimating CVD outcomes. This work highlights the need for more robust comparative standards when evaluating sleep-based AI models and demonstrating clinically relevant improvements over traditional risk scores. Beyond these baseline standards, other priorities to further improve sleep AI capabilities include appropriately accounting for age as a risk factor, more diverse data collection across age and healthy sleep, and leveraging explainability techniques to discover new features of sleep physiology that estimate CVD risk better than demographics alone.