GAN-based heterogeneous network for ancient mural restoration

Abstract Ancient murals, invaluable cultural artifacts, frequently suffer damage from environmental and human factors, necessitating effective restoration techniques. Traditional methods, which rely on manual skills, are time-consuming and often inconsistent. This study introduces an innovative mural restoration approach using a generative adversarial network (GAN) within a UNet architecture. The generator integrates Transformer and convolutional neural network (CNN) components, effectively capturing and reconstructing complex mural features. This work's novelty lies in integrating the Group-wise Multi-scale Self-Attention (GMSA), an Encoder-Decoder Feature Interaction (EDFI) module, and a Local Feature Enhancement Block (LFEB). These components allow the model to better capture, reconstruct, and enhance mural features, leading to a significant improvement over traditional restoration methods. Tested on a dataset of Tang Dynasty murals, the method demonstrated superior performance in PSNR, SSIM, and LPIPS metrics compared to seven other techniques. Ablation studies confirmed the effectiveness of the heterogeneous network design and the critical contributions of the GMSA, EDFI, and LFEB modules. Practical restoration experiments showed the method's ability to handle various types of mural damage, providing seamless and visually authentic restorations. This novel approach offers a promising solution for the digital preservation and restoration of cultural heritage murals, with potential applications in practical restoration projects.