MULTI-MODAL BREAST CANCER CLASSIFICATION USING DUAL-ATTENTION VISION TRANSFORMERS AND METADATA-AWARE FUSION

Breast cancer remains a leading cause of cancer mortality among women worldwide. For this reason, accurate and automated diagnostic tools are required. Mammography is regarded as the clinical gold standard for screening. However, diagnostic performance is limited by interpretive variability and time intensive analysis. Therefore, improved computational support is required. In this study, a novel multi modal deep learning approach is proposed, which jointly uses imaging data and patient metadata for breast cancer classification. Image features are extracted through a Dual Attention Vision Transformer named DaViT Tiny. Furthermore, structured clinical metadata, which include breast density, implant status, biopsy history, and tumor invasiveness, are processed through a multilayer perceptron. Then, a cross attention fusion module is applied, in which inter modal representations are aligned and weighted. As a result, complementary relationships between imaging features and clinical attributes are learned. The network is initialized through transfer learning on the CheXpert dataset. After that, fine tuning is performed on the RSNA mammography dataset, which contains more than 8000 labeled cases. The proposed model achieves an accuracy of 95.47, a sensitivity of 0.9655, and a specificity of 0.9439 on the test set. Consequently, performance exceeds that of single modality baselines. These results demonstrate that attention based fusion of imaging data and clinical information improves diagnostic precision and robustness. Hence, a scalable direction for early breast cancer detection is supported. Moreover, broader application to other medical imaging tasks is indicated for future research.