NoiseRater: Meta-Learning Framework for Adaptive Noise Valuation in Diffusion Models
A recent research publication presents NoiseRater, a meta-learning framework aimed at assessing instance-level noise during the training of diffusion models. The authors question the traditional belief that all injected noise carries the same level of information. They introduce a parametric noise rater that provides importance scores for specific noise realizations based on data and timestep, allowing for adaptive reweighting of the training objective. Utilizing bilevel optimization, the rater is trained to enhance downstream validation performance following inner-loop diffusion updates. An efficient two-stage pipeline is established, shifting from soft weighting in meta-training to hard noise selection in standard training. Comprehensive experiments on FFHQ and ImageNet reveal that not every noise sample is equally beneficial.
Key facts
- NoiseRater is a meta-learning framework for instance-level noise valuation in diffusion model training.
- It challenges the assumption that injected noise is uniformly informative.
- A parametric noise rater assigns importance scores to noise realizations conditioned on data and timestep.
- The rater is trained via bilevel optimization to improve downstream validation performance.
- A decoupled two-stage pipeline transitions from soft weighting to hard noise selection.
- Experiments were conducted on FFHQ and ImageNet datasets.
- Results show that not all noise samples contribute equally.
- The paper is available on arXiv with ID 2605.08144.
Entities
Institutions
- arXiv