Memory-efficient transformer network with feature fusion for breast tumor segmentation and classification task

The analysis of breast cancer using Ultrasounds, Magnetic resonance imaging (MRI), and Mammogram images plays a crucial role in the early detection of breast tumors in women. Nevertheless, accurately segmenting breast tumors remains a challenging task due to the presence of random variations, irregular shapes, and blurred boundaries within tumor regions. Blurred boundaries between breast tumor regions and healthy tissues may also cause higher rates of false-positives or incorrect segmentation results. Traditional convolutional neural networks (CNNs) are constrained by their inability to effectively capture global context information. As a result, they often yield unsatisfactory results in many scenarios. To address this, we have proposed a double encoder block to capture local and global context information related to breast tumor segmentation. In this model, we used a memory-efficient decoder block and a feature fusion block (FFB) to improve feature adaptation and classification capability among adjacent-level with the help of activating channels. Pyramid-dilated fusion (PDF) block based on depth-wise separable convolution is integrated within the transformer to achieve a comprehensive multiscale context. We enabled our model to suppress the irrelevant background noises of breast ultrasound and MRI images. Following this, we conducted comprehensive experiments to evaluate the effectiveness of our proposed Memory-efficient transformer network (MET-Net) for breast tumor segmentation and classification. To ensure reliable results, we employed ultrasounds and MRI datasets within a simulation environment. During the experiment analysis, the MET-Net model demonstrated extraordinary results in the presence of existing state-of-the-art CNNs models in terms of comparative measures. Source code of this work is available at https://github.com/ahmedeqbal/MET-Net.