Research on the virtual restoration of faded Dunhuang murals with a global attention mechanism

The Mogao Grottoes, located at the western end of the Hexi Corridor in Dunhuang, constitute a splendid artistic treasure trove of ancient Chinese civilization. However, due to various external factors, such as the environment and human activities, existing murals commonly suffer from fading and discoloration problems. The utilization of color restoration techniques grounded in deep learning promise perpetual preservation of mural images. Nevertheless, it is challenging to acquire a substantial number of authentic fading images and corresponding reference images as paired data for training, which constrains the scope of research and development in the domain of mural color restoration. The images generated by the current color restoration methods based on cycle generative adversarial networks suffer from poor semantic consistency, blurred edges, false colors and artifacts, and other mismatches with human visual perception. This paper proposes a novel approach of color restoration for unpaired mural images that uses cycle-consistent generative adversarial networks with an attention mechanism and a spectral normalization discriminator to address key challenges. First, to reduce the false colors and artifacts in the restored image caused by the insufficient extraction of mural detail features, a global attention mechanism based on a combination of average pooling and maximum pooling is constructed in the generator. This mechanism is designed to learn the effective information of the feature maps adaptively. In addition, we employ an SN-Patch discriminator to enhance the training stability and convergence speed of the model and improve the clarity of the color restored image. Finally, to further optimize the generated images, the network applies a composite loss function, a linear combination of adversarial loss, cycle consistency loss, identity mapping loss, and cyclic perceptual consistency loss, which contribute to improving texture quality and generating visually more natural color-restored images. The findings from the experiments conducted on both simulated and genuine faded mural images illustrate that the introduced approach outperforms others in terms of both objective and subjective evaluation standards, achieving a more precise recreation of the colors and intricate details present in the faded mural imagery.