A generative adversarial network (GAN)-based technique for synthesizing realistic respiratory motion in the extended cardiac-torso (XCAT) phantoms

Abstract Objective . Synthesize realistic and controllable respiratory motions in the extended cardiac-torso (XCAT) phantoms by developing a generative adversarial network (GAN)-based deep learning technique. Methods . A motion generation model was developed using bicycle-GAN with a novel 4D generator. Input with the end-of-inhale (EOI) phase images and a Gaussian perturbation, the model generates inter-phase deformable-vector-fields (DVFs), which were composed and applied to the input to generate 4D images. The model was trained and validated using 71 4D-CT images from lung cancer patients and then applied to the XCAT EOI images to generate 4D-XCAT with realistic respiratory motions. A separate respiratory motion amplitude control model was built using decision tree regression to predict the input perturbation needed for a specific motion amplitude, and this model was developed using 300 4D-XCAT generated from 6 XCAT phantom sizes with 50 different perturbations for each size. In both patient and phantom studies, Dice coefficients for lungs and lung volume variation during respiration were compared between the simulated images and reference images. The generated DVF was evaluated by deformation energy. DVFs and ventilation maps of the simulated 4D-CT were compared with the reference 4D-CTs using cross correlation and Spearman’s correlation. Comparison of DVFs and ventilation maps among the original 4D-XCAT, the generated 4D-XCAT, and reference patient 4D-CTs were made to show the improvement of motion realism by the model. The amplitude control error was calculated. Results . Comparing the simulated and reference 4D-CTs, the maximum deviation of lung volume during respiration was 5.8%, and the Dice coefficient reached at least 0.95 for lungs. The generated DVFs presented comparable deformation energy levels. The cross correlation of DVFs achieved 0.89 ± 0.10/0.86 ± 0.12/0.95 ± 0.04 along the x / y / z direction in the testing group. The cross correlation of ventilation maps derived achieved 0.80 ± 0.05/0.67 ± 0.09/0.68 ± 0.13, and the Spearman’s correlation achieved 0.70 ± 0.05/0, 60 ± 0.09/0.53 ± 0.01, respectively, in the training/validation/testing groups. The generated 4D-XCAT phantoms presented similar deformation energy as patient data while maintained the lung volumes of the original XCAT phantom (Dice = 0.95, maximum lung volume variation = 4%). The motion amplitude control models controlled the motion amplitude control error to be less than 0.5 mm. Conclusions . The results demonstrated the feasibility of synthesizing realistic controllable respiratory motion in the XCAT phantom using the proposed method. This crucial development enhances the value of XCAT phantoms for various 4D imaging and therapy studies.