Adaptive cascaded transformer U-Net for MRI brain tumor segmentation

Abstract Objective. Brain tumor segmentation on magnetic resonance imaging (MRI) plays an important role in assisting the diagnosis and treatment of cancer patients. Recently, cascaded U-Net models have achieved excellent performance via conducting coarse-to-fine segmentation of MRI brain tumors. However, they are still restricted by obvious global and local differences among various brain tumors, which are difficult to solve with conventional convolutions. Approach. To address the issue, this study proposes a novel Adaptive Cascaded Transformer U-Net (ACTransU-Net) for MRI brain tumor segmentation, which simultaneously integrates Transformer and dynamic convolution into a single cascaded U-Net architecture to adaptively capture global information and local details of brain tumors. ACTransU-Net first cascades two 3D U-Nets into a two-stage network to segment brain tumors from coarse to fine. Subsequently, it integrates omni-dimensional dynamic convolution modules into the second-stage shallow encoder and decoder, thereby enhancing the local detail representation of various brain tumors through dynamically adjusting convolution kernel parameters. Moreover, 3D Swin-Transformer modules are introduced into the second-stage deep encoder and decoder to capture image long-range dependencies, which helps adapt the global representation of brain tumors. Main results. Extensive experiment results evaluated on the public BraTS 2020 and BraTS 2021 brain tumor data sets demonstrate the effectiveness of ACTransU-Net, with average DSC of 84.96% and 91.37%, and HD95 of 10.81 and 7.31 mm, proving competitiveness with the state-of-the-art methods. Significance. The proposed method focuses on adaptively capturing both global information and local details of brain tumors, aiding physicians in their accurate diagnosis. In addition, it has the potential to extend ACTransU-Net for segmenting other types of lesions. The source code is available at: https://github.com/chenbn266/ACTransUnet .