VIPaint: Variational Inference for Diffusion-Based Image Inpainting
Researchers propose VIPaint, a hierarchical variational inference algorithm for image inpainting using pre-trained diffusion models. The method optimizes a non-Gaussian Markov approximation of the true diffusion posterior, enabling diverse high-quality imputations even for large masked regions. VIPaint outperforms existing baselines and is applicable to latent diffusion models, which generate high-quality images with lower computational cost. The approach works with state-of-the-art text-conditioned latent diffusion models, addressing limitations of prior methods that fail to produce samples from the true conditional distribution.
Key facts
- VIPaint uses hierarchical variational inference for diffusion-based inpainting.
- It optimizes a non-Gaussian Markov approximation of the diffusion posterior.
- The method works with latent diffusion models for high-quality, efficient generation.
- It handles large masked regions better than existing baselines.
- VIPaint is effective for text-conditioned latent diffusion models.
- The approach produces diverse imputations.
- It addresses the challenge of conditioning generative processes on corrupted images.
- The research is published on arXiv (2411.18929).
Entities
Institutions
- arXiv