RoAR-VAE: Improving Latent Diffusability through Robust Prediction

Key points

• Prior work improves latent diffusability through semantic alignment or joint tokenizer--diffusion training.1
• Revisiting REPA-E, a representative approach, we find that removing its diffusion objective and dedicated generative blocks while retaining semantic prediction from corrupted latents yields comparable downstream generation performance under a matched setting.1
• RobustAlign predicts pretrained semantic features from corrupted latents to preserve high-level structure, while RobustRecon reconstructs the clean image to retain fine-grained spatial and appearance information.1
• These results demonstrate that RoAR-VAE achieves a favorable balance between reconstruction fidelity and downstream generative learnability.1

Sources (2)

• [1]RoAR-VAE: Improving Latent Diffusability through Robust Prediction
  Hugging Face Daily Papers · Oct 19, 12:00 AM

  An effective visual tokenizer must preserve image fidelity while producing latents that downstream diffusion models can learn efficiently.

  Prior work improves latent diffusability through semantic alignment or joint tokenizer--diffusion training.

• [2]RoAR-VAE: Improving Latent Diffusability through Robust Prediction
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 11:42 AM · same content

Extractive summary: sentences quoted from the sources.