Amortized Guidance for Image Inpainting with Pretrained Diffusion Models
Researchers introduce Amortized Inpainting with Diffusion (AID), a method for image inpainting using pretrained diffusion models. Unlike existing approaches that train dedicated models or adapt per image, AID keeps the diffusion backbone fixed and trains a small reusable guidance module offline. This module is then applied across masked images without per-instance optimization. The problem is formulated as deterministic guidance with a supervised terminal objective, made learnable via an auxiliary Gaussian formulation. A continuous-time actor-critic algorithm learns the guidance module in a data-driven manner. Experiments on AFHQv2 and FFHQ under pixel EDM demonstrate effectiveness.
Key facts
- AID keeps a pretrained diffusion backbone fixed.
- A small guidance module is trained offline and reused.
- No per-instance optimization is required at deployment.
- Formulated as deterministic guidance with supervised terminal objective.
- Auxiliary Gaussian formulation makes the problem learnable.
- Continuous-time actor-critic algorithm is used for learning.
- Tested on AFHQv2 and FFHQ datasets.
- Method is a middle ground between task-specific models and per-image adaptation.
Entities
—