修补
计算机科学
人工智能
计算机视觉
嵌入
变压器
模式识别(心理学)
图像(数学)
量子力学
物理
电压
作者
Feng Li,Anqi Li,Jia Qin,Huihui Bai,Lin Wang,Runmin Cong,Yao Zhao
出处
期刊:IEEE transactions on computational imaging
日期:2022-01-01
卷期号:8: 743-758
被引量:3
标识
DOI:10.1109/tci.2022.3190142
摘要
Recent image inpainting methods have achieved remarkable improvements by using generative adversarial networks (GAN). Most of them have been designed to produce plausible results from high-level semantic features using only high-resolution (HR) supervision. However, because abundant details are lost in large holes, it is difficult to simultaneously synthesize details while preserving structural coherence in HR space. Besides, the correlations between the inside and outside of the missing region play a critical role in transferring relevant known information to generate semantic-coherent textures, especially in patch matching-based methods. In this work, we present SRInpaintor which inherits the merits of super-resolution (SR) and transformer for high-fidelity image inpainting. The SRInpaintor starts from global structure reasoning with low-resolution (LR) input and progressively refines the local textures in HR space, constituting a multi-stage framework with SR supervision. The bottom stage recovers coarse SR results that provide structural information as an appearance prior, and is combined with the higher-resolution corrupted image at the next stage to render available textures for the missing region. Such a design can analyse the image from LR to HR with the increase of stages, enabling coarse-to-fine information propagation and detail refinement. In addition, we propose a hierarchical transformer (HieFormer) to model the long-term correlations between distant contexts and holes. By embedding it into a compact latent space in a cross-scale manner, we can ensure reliable relevant texture transformation and robust appearance consistency. Experimental results demonstrate the superiority of our method compared with recent state-of-the-art methods. Code will be available on https://github.com/lifengshiwo/SRInpaintor.
科研通智能强力驱动
Strongly Powered by AbleSci AI