Imperfect Verifiers, Exact Targets: Inference-Time Scaling for Continuous Generation

Key points

• Verifier-guided inference-time scaling uses additional computation to score intermediate states and steer generation toward better outputs.1
• With correctly paired denoising and renoising transitions, the stationary distribution of CS-VGB at the clean checkpoint equals the desired reward-tilted target, even when intermediate verifiers are inaccurate.1
• In analytically tractable settings, we empirically demonstrate that CS-VGB recovers the desired reward-tilted distribution despite verifier error, whereas the compared verifier-based sampling methods fail to recover the target under the same verifier errors.1
• Across prompt-guided ImageNet generation, facial-attribute generation on CelebA, and multi-robot planning, recurrent CS-VGB search improves reward and task success as inference budgets grow, with image-generation rewards continuing beyond the observed plateaus of denoising-only baselines.1

Sources (1)

• [1]Imperfect Verifiers, Exact Targets: Inference-Time Scaling for Continuous Generation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 02:24 AM

  We introduce Continuous-State Verifier-Guided Backtracking (CS-VGB), an inference-time scaling method for sampling from reward-tilted distributions with continuous generative models through repeated denoising and renoising.

  Verifier-guided inference-time scaling uses additional computation to score intermediate states and steer generation toward better outputs.

Extractive summary: sentences quoted from the sources.