Risk-Averse Inference-Time Diffusion Model Alignment with Finite-Sample Guarantees

Key points

• Inference-time alignment steers diffusion models toward downstream objectives without costly fine-tuning, but existing methods typically optimize expected cost (or reward) without explicitly controlling tail risk.1
• We characterize the optimal path distribution and reverse transitions as functions of the CVaR threshold, reducing the alignment problem to a one-dimensional optimization problem over the threshold.1
• We show that this one-dimensional optimization problem can be nonconvex in general, establish sufficient conditions for convexity, and introduce quadratic regularization to enable efficient derivative-free ternary search.1
• Experiments on synthetic data and image generation using Stable Diffusion demonstrate improved tail performance over risk-neutral baselines while maintaining competitive average cost.1

Sources (1)

• [1]Risk-Averse Inference-Time Diffusion Model Alignment with Finite-Sample Guarantees
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 01:57 AM

  In this work, we propose a risk-averse framework for inference-time alignment of diffusion models that balances expected cost and Conditional Value-at-Risk (CVaR) under trajectory-level Kullback-Leibler regularization.

  Inference-time alignment steers diffusion models toward downstream objectives without costly fine-tuning, but existing methods typically optimize expected cost (or reward) without explicitly controlling tail risk.

Extractive summary: sentences quoted from the sources.