Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion

Key points

• Discrete diffusion models have become a powerful framework for modeling complex discrete distributions across a range of applications.1
• While theoretical convergence guarantees depend on consistency between the target and training distributions, such a case rarely holds in practice, making score mismatch a central challenge in discrete diffusion sampling.1
• This is particularly relevant to zero-shot conditional generation, where a model trained on the unconditional distribution is directly reused at inference time to sample from a conditional target distribution.1
• Based on this analysis, we propose a Randomized Hard Clamping (RHC) sampler for coordinatewise set constraints.1

Sources (1)

• [1]Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 01:05 AM

  In this paper, we provide the first theoretical analysis for score-mismatched discrete diffusion models and derive a general KL divergence upper bound that clearly identifies the additional bias caused by such a mismatch.

  Discrete diffusion models have become a powerful framework for modeling complex discrete distributions across a range of applications.

Extractive summary: sentences quoted from the sources.