An alternately optimized generative adversarial network with texture and content constraints for deformable registration of 3D ultrasound images

Objective.3D ultrasound non-rigid registration is significant for intraoperative motion compensation. Nevertheless, distorted textures in the registered image due to the poor image quality and low signal-to-noise ratio of ultrasound images reduce the accuracy and efficiency of the existing methods.Approach.A novel 3D ultrasound non-rigid registration objective function with texture and content constraints in both image space and multiscale feature space based on an unsupervised generative adversarial network based registration framework is proposed to eliminate distorted textures. A similarity metric in the image space is formulated based on combining self-structural constraint with intensity to strengthen the robustness to abnormal intensity change compared with common intensity-based metrics. The proposed framework takes two discriminators as feature extractors to formulate the texture and content similarity between the registered image and the fixed image in the multiscale feature space respectively. A distinctive alternating training strategy is established to jointly optimize the combination of various similarity loss functions to overcome the difficulty and instability of training convergence and balance the training of generator and discriminators.Main results.Compared with five registration methods, the proposed method is evaluated both with small and large deformations, and achieves the best registration accuracy with average target registration error of 1.089 mm and 2.139 mm in cases of small and large deformations, respectively. The performance on peak signal to noise ratio (PSNR) and structural similarity (SSIM) also proves the effective constraints on distorted textures of the proposed method (PSNR is 31.693 dB and SSIM is 0.9 in the case of small deformation; PSNR is 28.177 dB and SSIM is 0.853 in the case of large deformation).Significance.The proposed 3D ultrasound non-rigid registration method based on texture and content constraints with the distinctive alternating training strategy can eliminate the distorted textures with improving the registration accuracy.