Multimodal deep learning for personalized renal cell carcinoma prognosis: Integrating CT imaging and clinical data

BACKGROUND AND OBJECTIVE: Renal cell carcinoma represents a significant global health challenge with a low survival rate. The aim of this research was to devise a comprehensive deep-learning model capable of predicting survival probabilities in patients with renal cell carcinoma by integrating CT imaging and clinical data and addressing the limitations observed in prior studies. The aim is to facilitate the identification of patients requiring urgent treatment. METHODS: The proposed framework comprises three modules: a 3D image feature extractor, clinical variable selection, and survival prediction. Based on the 3D CNN architecture, the feature extractor module predicts the ISUP grade of renal cell carcinoma tumors linked to mortality rates from CT images. Clinical variables are systematically selected using the Spearman score and random forest importance score as criteria. A deep learning-based network, trained with discrete LogisticHazard-based loss, performs the survival prediction. Nine distinct experiments are performed, with varying numbers of clinical variables determined by different thresholds of the Spearman and importance scores. RESULTS: Our findings demonstrate that the proposed strategy surpasses the current literature on renal cancer prognosis based on CT scans and clinical factors. The best-performing experiment yielded a concordance index of 0.84 and an area under the curve value of 0.8 on the test cohort, which suggests strong predictive power. CONCLUSIONS: The multimodal deep-learning approach developed in this study shows promising results in estimating survival probabilities for renal cell carcinoma patients using CT imaging and clinical data. This may have potential implications in identifying patients who require urgent treatment, potentially improving patient outcomes. The code created for this project is available for the public on: GitHub.