Self-Supervised Representation-Guided Generative Dataset Distillation

A novel approach for dataset distillation, referred to as self-supervised representation-guided generative dataset distillation (SRG), has been introduced in an arXiv paper (2608.03218). This technique overcomes the shortcomings of current distillation methods, which generally focus on networks initialized randomly, while contemporary vision systems frequently utilize frozen pretrained encoders paired with lightweight adapters. SRG seeks to maintain the discriminative geometry of the pretrained representation space by converting self-supervised learning (SSL) geometry into diffusion guidance. It generates class-wise prototypes from real-image SSL representations and implements three objectives in SSL space: prototype alignment, inter-class discrimination, and intra-class assignment. During the diffusion sampling phase, SRG employs a stage-wise guidance approach, linking early denoising to the prototype's latent. This strategy aims to produce synthetic datasets that preserve the original data's effectiveness for downstream applications, especially when adapting frozen encoders. The paper is classified as a cross-type announcement and can be accessed via the provided arXiv link.