A Multiscale Spatial Transformer U-Net for Simultaneously Automatic Reorientation and Segmentation of 3-D Nuclear Cardiac Images

Accurate reorientation and segmentation of the left ventricular (LV) is essential for the quantitative analysis of myocardial perfusion imaging (MPI). This study proposes an end-to-end model, named as Multi-Scale Spatial Transformer UNet (MS-ST-UNet), which involves the multi-scale spatial transformer network (MSSTN) and multi-scale UNet (MSUNet) modules to perform simultaneous reorientation and segmentation of LV region from nuclear cardiac images. The multi-scale sampler produces images with varying resolutions, while scale transformer (ST) blocks are employed to align the scales of features. The proposed method is trained and tested using two different nuclear cardiac image modalities: 13N-ammonia Positron Emission Tomography (PET) and 99mTc-sestamibi Single Photon Emission Computed Tomography (SPECT). MS-ST-UNet attains Dice Similarity Coefficient (DSC) scores of 91.48% and 94.81% for PET LV myocardium (LV-MY) and SPECT LV-MY, respectively. Additionally, the mean square error (MSE) between predicted rigid registration parameters and ground truth decreases to below 1.4×10-2. The experimental findings indicate that the MS-ST-UNet yields notably reduced registration errors and more precise boundary detection for the LV structure compared to existing methods. This joint learning framework promotes mutual enhancement between reorientation and segmentation tasks, leading to cutting edge performance and an efficient image processing workflow.