Factorization-Guided Training and Inference for Diffusion-Drafted Speculative Decoding

Key points

• Diffusion drafters further parallelize proposal generation, but greater proposal capacity or higher mean accepted tokens (MAT) need not yield higher throughput.1
• We introduce DScout, which uses two complementary factorizations to guide diffusion drafter training and inference for higher throughput.1
• For inference, DScout factorizes throughput as the ratio of MAT to per-round latency, guiding target-conditioning depth, drafter depth, and inference block size selection to identify a high-throughput configuration for each workload.1
• Across six benchmarks, DScout achieves 3.84times and 4.14times mean speedups over autoregressive decoding on Qwen3-4B and Qwen3-8B, respectively, improving mean speedup over the state-of-the-art baseline by 9.1% and 7.1%.1

Sources (1)

• [1]Factorization-Guided Training and Inference for Diffusion-Drafted Speculative Decoding
  Hugging Face Daily Papers · Oct 7, 12:00 AM

  Speculative decoding accelerates large language model inference through parallel verification of tokens proposed by a drafter.

  Diffusion drafters further parallelize proposal generation, but greater proposal capacity or higher mean accepted tokens (MAT) need not yield higher throughput.

Extractive summary: sentences quoted from the sources.