Bi-Modal Transfer Learning for Classifying Breast Cancers via Combined B-Mode and Ultrasound Strain Imaging

Although accurate detection of breast cancer still poses significant challenges, deep learning (DL) can support more accurate image interpretation. In this study, we develop a highly robust DL model based on combined B-mode ultrasound (B-mode) and strain elastography ultrasound (SE) images for classifying benign and malignant breast tumors. This study retrospectively included 85 patients, including 42 with benign lesions and 43 with malignancies, all confirmed by biopsy. Two deep neural network models, AlexNet and ResNet, were separately trained on combined 205 B-mode and 205 SE images (80% for training and 20% for validation) from 67 patients with benign and malignant lesions. These two models were then configured to work as an ensemble using both image-wise and layer-wise and tested on a dataset of 56 images from the remaining 18 patients. The ensemble model captures the diverse features present in the B-mode and SE images and also combines semantic features from AlexNet and ResNet models to classify the benign from the malignant tumors. The experimental results demonstrate that the accuracy of the proposed ensemble model is 90%, which is better than the individual models and the model trained using B-mode or SE images alone. Moreover, some patients that were misclassified by the traditional methods were correctly classified by the proposed ensemble method. The proposed ensemble DL model will enable radiologists to achieve superior detection efficiency owing to enhance classification accuracy for breast cancers in ultrasound (US) images.