SCALE: Improving Guidance-Free Visual Generation via Self-Contrastive Alignment for Likelihood Estimation

Key points

• Classifier-free guidance (CFG) is essential for improving sample quality in visual generation, but it incurs substantial sampling overhead.1
• Existing guidance-free approaches halve this cost by approximating the CFG with a single model inference, but their performance is bounded by the CFG and inherits its limitations.1
• Integrating SCG into the standard likelihood objective via reparameterization yields Self-Contrastive Alignment for Likelihood Estimation (SCALE), a guidance-free training framework that produces SCG-enhanced predictions in a single forward pass.1
• Across class-conditional and text-to-image generation on diffusion and autoregressive models, SCALE consistently outperforms CFG-mimicking guidance-free baselines and improves further when combined with DPO, demonstrating that the self-contrastive signal yields gains complementary to both CFG and preference-based fine-tuning.1

Sources (1)

• [1]SCALE: Improving Guidance-Free Visual Generation via Self-Contrastive Alignment for Likelihood Estimation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 10:34 AM

  In this paper, we propose self-contrastive guidance (SCG), which contrasts the model's prediction on real and self-generated samples to obtain a direct, model-based signal that pushes generation toward the true data manifold.

  Classifier-free guidance (CFG) is essential for improving sample quality in visual generation, but it incurs substantial sampling overhead.

Extractive summary: sentences quoted from the sources.