EigDiff: Eigensample Diffusion - or how to significantly reduce training dataset size with marginal performance variation?

Key points

• Diffusion models are notoriously expensive to train, requiring large-scale datasets to learn how to gradually reverse the noise from images over multiple small denoising steps.1
• To mitigate the sensitivity to affine transformations of eigenimages, we harness the latent space of a pre-trained CLIP model.1
• We next compute latent eigenvectors by performing principal component analysis (PCA) in CLIP space.1
• Our comprehensive experiments with different diffusion models applied on multiple datasets indicate that our approach (EigDiff) can reduce training data by up to 14x and accelerate subset selection by up to 1607x, with marginal performance differences.1

Sources (1)

• [1]EigDiff: Eigensample Diffusion - or how to significantly reduce training dataset size with marginal performance variation?
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 10:36 PM

  In this paper, we explore the potential of harnessing eigensamples to train diffusion models on significantly smaller datasets, while preserving model performance.

  Diffusion models are notoriously expensive to train, requiring large-scale datasets to learn how to gradually reverse the noise from images over multiple small denoising steps.

Extractive summary: sentences quoted from the sources.