Predicting Lymph Node Metastasis of Lung Cancer: A Two-stage Multimodal Data Fusion Approach

Lung cancer is the leading cause of cancer death worldwide. Lymph node metastasis (LNM) status plays a vital role in determining the initial treatment for lung cancer patients, but it is difficult to diagnose accurately before surgery. Developing an LNM prediction model using multimodal data is the mainstream solution for this clinical problem. However, the current multimodal fusion methods may suffer from performance degradation when one type of modal data has poor predictive performance. In this study, we presented a two-stage multimodal data fusion approach to alleviate this problem. We first constructed unimodal prediction models using unimodal data separately and then used the encoders of the unimodal with frozen parameters as feature extractors and re-trained a new decoder to achieve the multimodal data fusion. We conducted experiments on real clinical multimodal data of 681 lung cancer patients collected from Peking University Cancer Hospital. Experimental results show that the proposed approach outperformed the state-of-the-art LNM prediction models and different multimodal fusion strategies. We conclude that the proposed method is a good option for multimodal data fusion when image data has poor discriminative performance.