Multimodal Distribution Matching for Vision-Language Dataset Distillation

A team of researchers has introduced Multimodal Distribution Matching (MDM), a framework that is sensitive to geometry for condensing extensive vision-language datasets into smaller synthetic collections. MDM tackles the issues of elevated computational expenses and the oversight of cross-modal relationships found in previous approaches by merging data, model, and loss-level elements. At the data stage, it generates synthetic image-text pairs through clustering within a joint embedding space. On the model front, it constructs a mixed teacher by blending fine-tuned models based on angular deviation from a pretrained anchor. The objective of this framework is to maintain the quality of representation and cross-modal alignment while adhering to stringent compute and memory constraints.