去模糊
修补
计算机科学
潜影
先验概率
扩散
反问题
投影(关系代数)
嵌入
潜变量
航程(航空)
算法
图像(数学)
人工智能
反向
贝叶斯概率
数学
图像处理
图像复原
数学分析
物理
材料科学
几何学
复合材料
热力学
作者
Hyungjin Chung,Jong Chul Ye,Peyman Milanfar,Mauricio Delbracio
出处
期刊:Cornell University - arXiv
日期:2023-10-02
标识
DOI:10.48550/arxiv.2310.01110
摘要
We propose a new method for solving imaging inverse problems using text-to-image latent diffusion models as general priors. Existing methods using latent diffusion models for inverse problems typically rely on simple null text prompts, which can lead to suboptimal performance. To address this limitation, we introduce a method for prompt tuning, which jointly optimizes the text embedding on-the-fly while running the reverse diffusion process. This allows us to generate images that are more faithful to the diffusion prior. In addition, we propose a method to keep the evolution of latent variables within the range space of the encoder, by projection. This helps to reduce image artifacts, a major problem when using latent diffusion models instead of pixel-based diffusion models. Our combined method, called P2L, outperforms both image- and latent-diffusion model-based inverse problem solvers on a variety of tasks, such as super-resolution, deblurring, and inpainting.
科研通智能强力驱动
Strongly Powered by AbleSci AI