Consistent Distribution Matching for Data-Free Diffusion Distillation

Key points

• Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration.1
• Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one-step or few-step generation.1
• We prove that minimizing our objective indicates Wasserstein convergence of the student flow-map pushforwards to the teacher marginals.1
• On ImageNet 256$\times$256, our method attains an FID of 2.04 with a single function evaluation (1-NFE) and a 4-NFE FID of 1.37 within 40 epochs of training, surpassing the state-of-the-art distillation baselines without data.1

Sources (2)

• [1]Consistent Distribution Matching for Data-Free Diffusion Distillation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 11:34 PM

  In this work, we propose Consistent Distribution Matching, a simulation-free and data-free distillation method for accelerating diffusion and flow models while preserving strong generative capacity.

  Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration.

• [2]Consistent Distribution Matching for Data-Free Diffusion Distillation
  Hugging Face Daily Papers · Oct 6, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.