Target Regeneration Shrinks the Gains of Draft-Training Objectives in Speculative Decoding
Speculative decoding accelerates large language models by verifying drafted tokens in parallel.
ProofPaper ↗
Key points
- New draft-training objectives are usually credited with the acceptance they add over cross-entropy on externally written responses, yet many pipelines let the target rewrite them.
- With a vision-language target and a block drafter, we train three objectives on four versions of one corpus: cross-entropy on corpus tokens, token-level distillation from the target, and distillation gated to positions that would survive verification.
- Distillation and the gate add large gains on external responses, but after greedy regeneration, distillation adds a smaller gain and the gate little or none.
- A second vision corpus shows the shrinkage, as does a text-only replication, for distillation also with an autoregressive drafter.
Sources (1)
- [1]Target Regeneration Shrinks the Gains of Draft-Training Objectives in Speculative DecodingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 12:26 AM
Speculative decoding accelerates large language models by verifying drafted tokens in parallel.
New draft-training objectives are usually credited with the acceptance they add over cross-entropy on externally written responses, yet many pipelines let the target rewrite them.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 21, 2026Think Harder Where It Matters: Item-Level Adaptive Recursion for Sequential Recommendation
- Oct 20, 2026Large Flow Language Models
- Oct 19, 2026DStill: Trajectory Consistency Distillation for Accelerated One-Step Drafting in Block Diffusion Speculative Decoding
- Oct 19, 2026Exposing Vulnerability Monoculture Through Systemic Adversarial Robustness
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
- Oct 19, 2026Kapitsa: open-source Apple Silicon optimized Metal engine built for Qwen and Nemotron. See benchmarks of Qwen 27B, Nemotron + 8 other models on an M1 Pro (16GB), M4 Pro (48GB) and M5 Max (128GB)