HyTransMA: A Hybrid Model Using Transformer with Multi-scale Attentions for Medical Image Segmentation

The emergence of deep learning provides a solution to reduce the burden on doctors and improve the accuracy of medical image segmentation. However, most of the methods face many challenges when applied in practice. This is mainly due to the small gray difference between the target region and the background, the lack of clarity of the target area boundary, and the continuous change of the target shape with contraction and diastole. In order to solve the above challenges, in this paper, we propose a medical image segmentation model and design a 3paths structure to achieve cross-level and multi-scale information fusion. At the same time, combining the advantages of Transformer and Convolutional Neural Network (CNN), the dual branch of Transformer and CNN is used in the feature extraction stage. Dual-branch independent data input introduces the idea of multitask learning. In the later stage of feature extraction, the feature maps of different scales and stages are fed into the Multi-headed Self-Attention (MSA) mechanism module to achieve global attention. A large number of experiments and analyses have shown that our proposed model outperforms many common models in this field, which has been verified on the abdominal multi-organ segmentation dataset and the heart segmentation dataset.