Learning $ω$-Powered Conditional Distributions with Importance-Weighted Flow Matching

Key points

• Classifier-Free Guidance (CFG) with guidance scale $γ$ is commonly motivated by the $ω$-powered conditional distribution obtained by setting $ω=γ$, but CFG does not, in general, sample from this distribution exactly.1
• With exact normalized weights, the resulting objective has the same gradient as flow matching on the powered target; in practice, we use self-normalized estimation and a regularized empirical approximation for high-dimensional data.1
• On synthetic distributions, IW-CFM recovers analytical powered targets for which numerically matched CFG, $γ=ω$, exhibits a visible mismatch.1
• On ImageNet, an amortized lightweight corrector achieves better alignment than fixed- and matched-guidance CFG with the corresponding weighted and filtered reference distributions.1

Sources (1)

• [1]Learning $ω$-Powered Conditional Distributions with Importance-Weighted Flow Matching
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 08:51 AM

  We introduce Importance-Weighted Conditional Flow Matching (IW-CFM), which reweights conditional flow-matching examples using density ratios obtained from frozen conditional and unconditional flow models.

  Classifier-Free Guidance (CFG) with guidance scale $γ$ is commonly motivated by the $ω$-powered conditional distribution obtained by setting $ω=γ$, but CFG does not, in general, sample from this distribution exactly.

Extractive summary: sentences quoted from the sources.