Whole-body CT-to-PET synthesis using a customized transformer-enhanced GAN

Abstract Background . Positron emission tomography with 2-deoxy-2-[fluorine-18]fluoro-D-glucose integrated with computed tomography (18F-FDG PET-CT) is a multi-modality medical imaging technique widely used for screening and diagnosis of lesions and tumors, in which, CT can provide detailed anatomical structures, while PET can show metabolic activities. Nevertheless, it has disadvantages such as long scanning time, high cost, and relatively high radiation doses. Purpose . We propose a deep learning model for the whole-body CT-to-PET synthesis task, generating high-quality synthetic PET images that are comparable to real ones in both clinical relevance and diagnostic value. Material . We collect 102 pairs of 3D CT and PET scans, which are sliced into 27 240 pairs of 2D CT and PET images (training: 21,855 pairs, validation: 2810 pairs, testing: 2575 pairs). Methods . We propose a transformer-enhanced generative adversarial network (GAN) for whole-body CT-to-PET synthesis task. The CPGAN model uses residual blocks and fully connected transformer residual blocks to capture both local features and global contextual information. A customized loss function incorporating structural consistency is designed to improve the quality of synthesized PET images. Results . Both quantitative and qualitative evaluation results demonstrate effectiveness of the CPGAN model. The mean and standard variance of NRMSE, PSNR and SSIM values on test set are ( 16.90 ± 12.27 ) × 10 − 4 , 28.71 ± 2.67 and 0.926 ± 0.033 , respectively, outperforming other seven state-of-the-art models. Three radiologists independently and blindly evaluated and gave subjective scores to 100 randomly chosen PET images (50 real and 50 synthetic). By Wilcoxon signed rank test, there are no statistical differences between the synthetic PET images and the real ones. Conclusions . Despite the inherent limitations of CT images to directly reflect biological information of metabolic tissues, CPGAN model effectively synthesizes satisfying PET images from CT scans, which has potential in reducing the reliance on actual PET-CT scans.