A foundation model for predicting outcomes of neoadjuvant chemotherapy in breast cancer

Background: Although neoadjuvant chemotherapy (NAC) is a widely adopted approach in the treatment of breast cancer, personalizing the intensity of subsequent adjuvant therapy remains a major clinical challenge due to tumor heterogeneity and the lack of reliable biomarkers. Existing strategies fall short in identifying patients who could benefit from intensive adjuvant chemotherapy, particularly among non-pCR cases. To address this, we developed a foundation model that integrates histopathology and clinical data to support individualized treatment decisions. Methods: We collected whole-slide images and clinical data from 1,543 patients with non-metastatic invasive breast cancer who underwent planned NAC prior to surgery, across three cohorts: training cohort (n = 756), validation cohort (n = 560), and test cohort (n = 227). A hybrid AI-pathology model was developed, combining a convolutional neural networks (CNN) branch and a transformer-based foundation model pretrained on The Cancer Genome Atlas (TCGA) whole-slide images. This architecture enabled robust feature extraction from histopathological slides. These features were integrated with clinical data in a multimodal framework to predict pathological complete response (pCR) and disease-free survival (DFS). Results: The proposed hybrid AI-multimodal model demonstrated superior performance for pCR, achieving an AUC of 0.999 in both the training and validation cohorts. Integration with clinicopathological data yielded a multimodal model with enhanced predictive accuracy for both pCR and DFS, showing high AUCs across all cohorts, including 0.994 for pCR and 0.885 for 4-year DFS in the blinded test cohort. Notably, among non-pCR patients classified as high- or medium-risk by the AI-multimodal model, significant differences in DFS were observed across those who received intensive, standard, or no adjuvant chemotherapy. Furthermore, visualization heatmaps offered interpretability by linking model predictions to tumor microenvironment features, providing insights into the biological basis of treatment response. Conclusion: This foundation model integrating histopathological and clinical data enables accurate prediction of NAC response and long-term outcomes in breast cancer. Beyond prognostication, it offers a clinically actionable tool for identifying high-risk non-pCR patients who may benefit from intensive adjuvant chemotherapy, advancing precision oncology in the neoadjuvant setting.