PV-Decoding: Progressive Verification Decoding with Decode Gating for Diffusion Language Models

Key points

• Discrete diffusion language models (dLLMs) offer an attractive alternative to autoregressive generation by updating multiple tokens in parallel, but their inference efficiency is still limited by the cost of safely committing tentative predictions.1
• In particular, speculative decoding for dLLMs is challenging because speculative tokens within the same diffusion block may depend on one another, making aggressive acceptance prone to error propagation and strict verification computationally expensive.1
• PV-Decoding partitions masked positions into directly decoded tokens, speculative tokens, and unresolved tokens using two confidence thresholds, and then verifies speculative tokens with progressively stronger policies, including single-batch, dual-batch, and multi-batch verification.1
• Experiments on three diffusion language models across GSM8K, MBPP, and HumanEval show that PV-Decoding reduces sequential model forward passes (MFPs) by 8%--37% relative to confidence-based decoding, while keeping the largest observed accuracy drop to 1.3%.1

Sources (1)

• [1]PV-Decoding: Progressive Verification Decoding with Decode Gating for Diffusion Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 02:34 PM

  We present PV-Decoding, a confidence-driven speculative decoding framework with progressive verification for block diffusion inference.

  Discrete diffusion language models (dLLMs) offer an attractive alternative to autoregressive generation by updating multiple tokens in parallel, but their inference efficiency is still limited by the cost of safely committing tentative predictions.

Extractive summary: sentences quoted from the sources.