Enhanced image restoration via dual-module U-Net: integrating multi-scale spatial and frequency features

Abstract Image restoration aims to recover original images from degraded versions, a task challenged by the need to capture multi-scale features and utilize frequency-domain information. Traditional diffusion models, while adaptable, often fall short in feature modeling, limiting restoration performance. This paper introduces DMU-Net, integrating spatial-frequency domain dual-dimensional feature enhancement into a diffusion model-based Conditional U-Net. We propose the Multi-Scale Feature Aggregation Module and the Nonlocal-Frequency Multi-Scale Attention Module to strengthen cross-scale feature correlations and capture long-range dependencies in the frequency domain. Experiments on image deraining (Rain100H), image inpainting (CelebA-HQ 256$\times $256), and 4$\times $ image super-resolution (DIV2K) demonstrate that DMU-Net achieves strong and competitive performance across multiple image restoration tasks, with particularly notable advantages in perceptual quality and restoration accuracy for complex degraded images. Our findings offer a new technical approach for general image restoration tasks.