ViGNet: A clinical data-supported deep learning approach for NSCLC immunotherapy response prediction in digital pathology

Histopathology is the cornerstone of oncology diagnosis, while whole-slide images (WSIs) enable the transition to digital, quantitative pathology. Leveraging WSIs to accurately predict therapeutic response is increasingly vital for advancing precision oncology and optimizing clinical workflows. However, existing artificial intelligence models for WSI-based immunotherapy response prediction often yield suboptimal performance, as they frequently fail to fully capture the complex multi-scale morphological features and heterogeneous information inherent in clinical pathology data. To address this challenge, we propose ViGNet, a Visual-Global Relation Fusion Network designed for immunotherapy response prediction directly from WSIs. ViGNet is a multimodal framework that effectively integrates histopathological image features with specific clinical data modalities, including gene expression profiles and cancer type text. The proposed architecture features two primary encoders: a multi-scale visual encoder and a gene-driven encoder. The visual encoder utilizes pre-trained CTransPath and an MSCNNPath to extract pyramidal image features, which are integrated with cancer-type diagnostic priors. Meanwhile, the gene-driven encoder converts gene-guided information into a slide-level global relational prior. This global relational prior was then injected into the patch-level multi-scale visual representation learning via a top-down SRFF module, and the resulting representations were subsequently passed to a classification head to predict immunotherapy response. Across one internal and two independent external cohorts, ViGNet achieves 82.55% discrimination ability for immunotherapy response prediction, outperforming the baseline methods. These results underscore ViGNet's potential as a clinical decision-support tool, aiding personalized treatment planning and enabling informed clinical decision-making through accurate and interpretable predictions.