An enhanced multiscale generation and depth-perceptual loss-based super-resolution network for prostate ultrasound images

Abstract Computer-aided diagnosis of prostate ultrasound images is gradually being combined with deep learning to help detect and treat diseases. However, ultrasound images of the prostate have problems such as low resolution and unbalanced categories. In addition, the current image classification algorithms have difficulty with classification network performance due to insufficient data volume. To solve these problems, inspired by CycleGAN, we propose an enhanced multiscale generation and depth-perceptual loss-based super-resolution (SR) network for prostate ultrasound images (EGDL-CycleGAN). We study and improve the generative network and perceptual loss of CycleGAN. In this work we achieve multiscale feature extraction through an improved generator, and utilize full-scale skip connections between encoder and decoder to capture fine-grained details and coarse-grained semantics at full scale. This effectively improves the performance of the generative network and makes the reconstruction effect better. We also use the residual structure for deep extraction of features to obtain perceptual loss, and add this to the network loss function for training the model. This enables the model to learn the global and local differences between the real and generated images. This approach pays more attention to the edge information and spatial information of the image, and provides relevant spatial information feedback to the generator to improve the generator’s ability to perceive consistent super-resolution. The method can enhance the prostate ultrasound image dataset and provide rich images for the next step in intelligence-assisted classification and diagnosis of prostate cancer ultrasound images. The evaluation of peak signal-to-noise ratio/structural similarity and visual effects against the benchmark of our datasets illustrates that our proposed approach is effective and superior to the bicubic classic image SR reconstruction algorithm, the SRGAN perception-driven method and the CycleGAN method applied to ultrasound images. In addition, the method of using the original dataset combined with the SR reconstruction image dataset can effectively improve the accuracy of the classification network in intelligence-assisted classification diagnosis of prostate cancer ultrasound images. In EfficientNetV2 the accuracy is improved from 0.843 to 0.867 and in Swin Transformer the accuracy is improved from 0.893 to 0.917.