An Efficient Survival Prediction Framework Based on Multimodal Multi-Instance Fuzzy Optimal Transport

Accurately predicting the survival time of cancer patients is of significant clinical importance for oncologists in evaluating disease progression and treatment efficacy. In recent years, integrating multimodal data (such as pathological images and genomic data) has played a key role in understanding the complexity and heterogeneity of personalized cancer treatment, as well as improving survival prediction performance. However, these studies still face several challenges: they often overlook the fuzzy uncertainties in cross-modal interactions and the high computational complexity of pathological images. To address these issues, we propose an efficient cancer survival prediction framework based on multimodal multi-instance fuzzy optimal transport. This framework aims to capture fuzzy correlations between global latent structures and effectively integrate complex interactions among different modalities. Specifically, we first employ a fuzzy clustering algorithm to cluster patches extracted from all whole slide images of the same patient into different phenotypes, thereby reducing computational complexity. Next, we introduce a hierarchical alternating encoding paradigm for alternating encoding of intramodal and cross-modal feature representations. Simultaneously, to capture fuzzy correlations between global latent structures, we design a fuzzy optimal transport-based bilateral cross-attention module. This module introduces fuzzy membership into the global optimal matching, modeling the fuzzy interactions between histology and genomics, thereby facilitating the transmission of complementary multimodal information. Subsequently, we perform multimodal feature fusion to obtain the final survival prediction. Finally, experiments conducted on five cancer datasets demonstrate that our method achieves significant improvements in cancer survival prediction.