TeCoP: Accelerating Diffusion Large Language Model Decoding via Tentative Context Pipelining
Diffusion large language models (dLLMs) predict multiple token positions per forward pass, yet block-wise decoders serialize generation and repeatedly recompute stable predictions.
ProofPaper ↗
Key points
- We attribute this inefficiency to coupling a prediction's readiness for irreversible commitment with its utility as context.
- We introduce Tentative Context Pipelining (TeCoP), a training-free decoder that decouples these roles through staggered block-parallel pipelining and draft-and-verify.
- TeCoP reuses stable but uncommitted predictions as provisional cross-block context, with commitment governed by confidence and stability checks.
- Across four LLaDA and Dream dLLMs on GSM8K, Minerva MATH, and MBPP, TeCoP reduces number of forward evaluations by up to 87% and achieves up to 7.50-fold throughput without KV caching and 11.77-fold with caching over the uncached original decoder.
Sources (1)
- [1]TeCoP: Accelerating Diffusion Large Language Model Decoding via Tentative Context PipeliningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:48 AM
Diffusion large language models (dLLMs) predict multiple token positions per forward pass, yet block-wise decoders serialize generation and repeatedly recompute stable predictions.
We attribute this inefficiency to coupling a prediction's readiness for irreversible commitment with its utility as context.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Large Flow Language Models
- Oct 19, 2026Profile-DP: Differential Privacy for User-Controlled Profile Release in Personalized LLM Inference
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
- Oct 19, 2026NVIDIA Cosmos3-H-Dreams: Bringing Clinical Surgery to Life Through Generative Simulation
- Oct 19, 2026Balance-Aware Pivot Selection for Parallel Diffusion LLM Decoding
- Oct 19, 2026Keep the Lineages Alive: Diversity-Preserving SMC for Diffusion LLM Decoding