Integrating images and genomics for multi-modal cancer survival analysis via mixture of experts

Survival prediction seeks to provide patient prognosis by measuring the time span from diagnosis or the first treatment to the occurrence of a specific event of interest. It represents a challenging ordinal regression task that often involves modeling the intricate interactions among multiple data modalities, such as genomic profiles and Whole Slide Images (WSIs). Despite recent advancements, two critical obstacles persist: (i) learning effective representations for each modality, and (ii) capturing the intricate interactions and heterogeneity among different features. To address these challenges, we propose SurMoE ( Sur vival analysis with M ixture o f E xperts), a novel framework that designs a Mixture of Experts (MoE) architecture for multi-modal survival prediction. Specifically, we introduce a patch clustering layer to identify morphological prototypes from the vast collection of WSI patches and incorporate gene set enrichment analysis to capture biological associations among pathways and gene sets, yielding more robust modality representations. To model the intrinsic relationships within pathological WSIs and genomic profiles, we integrate multiple experts that dynamically adapt to diverse input patterns through a routing mechanism. Additionally, we employ cross-modal attention to seamlessly integrate multi-modal data and introduce a self-attention pooling module to refine modality-specific insights, thereby enhancing the accuracy of survival prediction. We conduct extensive experiments on five public TCGA datasets, supplemented by ablation studies, statistical analyses, and visualizations. The results demonstrate that our proposed SurMoE outperforms existing State-Of-The-Art (SOTA) methods, with an average increase of 2.29% in C-index across all datasets compared to the SOTA, underscoring the potential of MoE layer for refined multi-modal survival outcome prediction. Our code will be released at https://github.com/coffeeNtv/SurMoE .