StitchVM: Aligning Diffusion Models via Stitched Value Functions

StitchVM is a framework designed for model stitching that adeptly shifts reward models trained on pristine images into noisy latent spaces for the purpose of diffusion alignment. This innovation tackles the difficulty of synchronizing diffusion-based generative models with specific task rewards, as rewards are established for clean outputs, yet alignment necessitates value assessments in noisy intermediate latents. While current approaches like Tweedie estimates are efficient, they carry bias, and Monte Carlo estimates, though more precise, demand significant computational resources. StitchVM builds upon an existing reward model, modifying it to provide a viable solution. The research has been made available on arXiv under ID 2605.19804.