VaeZip: store VAE-encoded latent datasets in 5-10x fewer space with almost no quality loss

Key points

• This is mostly because jpeg and video codecs do there job and compress data very well, while latents are almost uncompressed.1
• I want to give a heads-up right away that this thing have very narrow use case when your latent dataset is large enough that it doesn't fit your local disk, so you have to use some s3 storage which may cost a lot when the dataset is so large.1
• Anyway, VaeZip is a PyTorch-based library that implements several latent quantization codecs and stores the compressed tensors in common Safetensors files that are much smaller then if you store these tensors in fp32.1
• That allows to store latent tensor with about 3 bits/value precision (10x smaller files) while keeping pixel MSE at 0.0003 after decoding.1

Sources (1)

• [1]VaeZip: store VAE-encoded latent datasets in 5-10x fewer space with almost no quality loss
  r/StableDiffusion (top, day) · Oct 16, 11:39 PM

  This is VAE latent files that take much more space than original data, while VAE should compress data, not making it more heavy.

  This is mostly because jpeg and video codecs do there job and compress data very well, while latents are almost uncompressed.

Extractive summary: sentences quoted from the sources.