Reminder: try probabilistic MTP if you missed it. Decode +14% on prose
Optimal draft-n-max / draft-p-min seem to be in line with greedy sampling.
Proof1 community thread
Key points
- Update your llama if you haven't done so yet.
- Main gain seems to be on prose generation.
- Tests above ran with thinking off, ngram-mod off.
Sources (1)
- [1]Reminder: try probabilistic MTP if you missed it. Decode +14% on proser/LocalLLaMA (top, daily) · Oct 11, 09:20 AM
Optimal draft-n-max / draft-p-min seem to be in line with greedy sampling.
Update your llama if you haven't done so yet.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 10, 202648Gb VRAM speed AND quality ! (Qwen 3.8 27B Swift 1.5 W8A16)
- Oct 10, 2026Qwen3.8-27B on a single 3090: 140 tok/s on code with a custom megakernel
- Oct 10, 2026Improve token per second without touching quant
- Oct 10, 2026Strata with Qwen3.8 Flash Next UD-Q4_K_XL
- Oct 8, 2026ollama/ollama v0.40.2
- Oct 8, 2026unslothai/unsloth v0.1.905-beta: Sandboxing is here!