Multimodal data fusion for cancer biomarker discovery with deep learning

Technological advances have made it possible to study a patient from multiple angles with high-dimensional, high-throughput multiscale biomedical data. In oncology, massive amounts of data are being generated, ranging from molecular, histopathology, radiology to clinical records. The introduction of deep learning has greatly advanced the analysis of biomedical data. However, most approaches focus on single data modalities, leading to slow progress in methods to integrate complementary data types. Development of effective multimodal fusion approaches is becoming increasingly important as a single modality might not be consistent and sufficient to capture the heterogeneity of complex diseases to tailor medical care and improve personalized medicine. Many initiatives now focus on integrating these disparate modalities to unravel the biological processes involved in multifactorial diseases such as cancer. However, many obstacles remain, including lack of usable data as well as methods for clinical validation and interpretation. Here, we cover these current challenges and reflect on opportunities through deep learning to tackle data sparsity and scarcity, multimodal interpretability and standardization of datasets. Cancer diagnosis and treatment decisions often focus on one data source. Steyaert and colleagues discuss the current status and challenges of data fusion, including electronic health records, molecular data, digital pathology and radiographic images, in cancer research and translational development.