Cross‐attention multimodal fusion for EGFR mutation prediction: A predictive model using CT and whole‐slide images

BACKGROUND: The prediction of Epidermal Growth Factor Receptor (EGFR) mutation status in advanced lung adenocarcinoma is crucial for targeted therapy. Since EGFR mutations manifest as both macroscopic imaging features on CT and microscopic morphological changes in tissue, integrating these multiscale signals is essential for a comprehensive diagnostic assessment. However, current related research faces two key limitations: on one hand, unimodal deep learning models suffer from limited representational power; on the other hand, existing multimodal methods fail to address the inherent data structural discrepancies between continuous CT and discrete WSI, often losing critical fine-grained details due to forced data compression or shared semantic bottlenecks. OBJECTIVE: To address the above limitations and improve the reliability of EGFR mutation status prediction, this study aims to propose a novel multimodal fusion framework (MFCA) that can effectively capture cross-modal semantic interactions and align imaging features across different scales. METHODS: A novel MFCA based on Cross-Attention (MFCA) is proposed, and its implementation steps are as follows: 1. First, a region-of-interest-guided approach is utilized to coarsely segment whole-slide histopathology images (WSI) into three constituent regions, namely cancerous, stromal, and other regions; 2. Then, a dual-branch encoder is employed to separately extract features from two types of imaging data-global features from Computed Tomography (CT) scans and region-specific features from the segmented WSI; 3. Critically, a bidirectional cross-attention module is introduced into the framework, which is designed to facilitate deep semantic interaction and alignment between the macroscopic context of CT imaging and the microscopic context of histopathology, thereby achieving highly efficient and discriminative feature fusion. RESULTS: On the external validation set, our MFCA framework achieved robust performance, with Area Under the Curve (AUC) values of 0.758(95% CI: 0.683-0.832) for cancerous regions, 0.805(95% CI: 0.716-0.900) for stromal regions, and 0.760(95% CI: 0.686-0.833) for other regions. The model's performance, particularly in the stromal component, was statistically superior to all baseline and competing models. CONCLUSION: The proposed MFCA framework predicts EGFR mutation status by innovatively integrating macroscopic CT imaging with region-specific microscopic WSI features. It serves as a valuable computational tool to support precision oncology for patients with advanced lung adenocarcinoma.