Overall survival time prediction for glioblastoma using multimodal deep KNN

Glioblastoma (GBM) is a severe malignant brain tumor with bad prognosis, and overall survival (OS) time prediction is of great clinical value for customized treatment. Recently, many deep learning (DL) based methods have been proposed, and most of them build deep networks to directly map pre-operative images of patients to the OS time. However, such end-to-end prediction is sensitive to data inconsistency and noise. In this paper, inspired by the fact that clinicians usually evaluate patient prognosis according to previously encountered similar cases, we propose a novel multimodal deep KNN based OS time prediction method. Specifically, instead of the end-to-end prediction, for each input patient, our method first search itsKnearest patients with known OS time in a learned metric space, and the final OS time of the input patient is jointly determined by theKnearest patients, which is robust to data inconsistency and noise. Moreover, to take advantage of multiple imaging modalities, a new inter-modality loss is introduced to encourage learning complementary features from different modalities. The in-house single-center dataset containing multimodal MR brain images of 78 GBM patients is used to evaluate our method. In addition, to demonstrate that our method is not limited to GBM, a public multi-center dataset (BRATS2019) containing 211 patients with low and high grade gliomas is also used in our experiment. As benefiting from the deep KNN and the inter-modality loss, our method outperforms all methods under evaluation in both datasets. To the best of our knowledge, this is the first work, which predicts the OS time of GBM patients in the strategy of KNN under the DL framework.