Identify Consistent Imaging Genomic Biomarkers for Characterizing the Survival-Associated Interactions Between Tumor-Infiltrating Lymphocytes and Tumors

The tumor-infiltrating lymphocytes (TILs) and its correlation with tumors play a critical role in the development and progression of breast cancer. Existing studies have demonstrated that the combination of the whole-slide pathological images (WSIs) and genomic data can better characterize the immunological mechanisms of TILs and assess the prognostic outcome in breast cancer. However, it is still very challenging to characterize the intersections between TILs and tumors in WSIs because of their large size and heterogeneity patterns, and the high dimensional genomic data also brings difficulty for the integrative analysis with WSIs data. To address the above challenges, in this paper, we propose an interpretable multi-modal fusion framework, IMGFN, that can fuse the interaction information between TILs and tumors with the genomic data via an attention mechanism for prognosis predictions of breast cancer. Specifically, for WSIs data, we use the graph attention network (i.e., GAT) to describe the spatial interactions of TILs and tumor regions across WSIs. As to genomic data, we use co-expression network analysis algorithms to cluster genes into co-expressed modules followed by applying the Concrete Autoencoders to select survival-associated modules. Finally, a self-attention layer is adopted to combine both the imaging and genomic features for the prognosis prediction of breast cancer. The experimental results on The Cancer Genome Atlas(TCGA) dataset suggest that the proposed IMGFN can not only achieve better prognosis results than the comparing methods but also identify consistent survival-associated imaging and genomic biomarkers correlated strongly with the interaction between TILs and tumors.