Drifting Models Linked to Score-Based Generative Modeling

A recent study reveals a clear relationship between drifting models and score-based generative modeling. Drifting models utilize one-step generators, optimizing a kernel-induced mean-shift discrepancy between the distributions of data and models, typically employing Laplace kernels. This discrepancy assesses kernel-weighted movements toward adjacent data and model samples, establishing a transport direction. The research indicates that for Gaussian kernels, the population mean-shift field corresponds to the difference in scores (gradient-log-densities) between Gaussian-smoothed data and model distributions, as outlined by Tweedie's formula. This finding connects to the score-matching principle that underpins diffusion models, thereby merging two methodologies in generative modeling.