Breast cancer diagnosis with MFF-HistoNet: a multi-modal feature fusion network integrating CNNs and quantum tensor networks

Prompt diagnosis of breast malignancy is crucial for treatment and patient survival. Computer-aided diagnosis (CAD) technology can improve efficiency, accuracy, and treatment options. The existing algorithms for classifying breast cancer histopathological images have limitations, including high parameter counts, ineffective extraction of global features, and substantial time costs, which result in the loss of valuable information. This study proposes a robust Multi-Modal Feature Fusion Network for Histopathology (MFF-HistoNet) to address the multi-grading challenges of breast image and significantly boost diagnostic accuracy. MFF-HistoNet combines a CNN and a Quantum Tensor Network (QTN), which reduces model parameters through parameter compression, enabling deeper global features. The data enhancement method ensures a balanced training set and minimizes color interference. The GLCM method is fused with LBP and Gabor filtering to obtain local cell shape characteristics of histopathological images in space and different scales and directions. Leveraging the BreaKHis dataset, MFF-HistoNet differentiates between eight breast cancer subtypes and reduces model complexity while preserving the ability to capture vital spatial relationships, thus enhancing computational efficiency. The MFF-HistoNet algorithms reveal the benchmark performance, achieving impressive accuracy of 98.8% at the image level and 98.4% at the patient level under 100 × magnification and 98.1% and 98.9% under 40 × magnification, outperforming existing models and reducing resource requirements. The Grad-CAM method proves the fusion model's reliability and interoperability, showing its firm resolution and good performance. The proposed model codes are publicly available at: https://github.com/tmsherazi/MFF-HistoNet with DOI: https://doi.org/10.5281/zenodo.14808037 .