Multi-token prediction

Timeline

1. Oct 19, 2026 · Opinion / analysis · 1 source
   incoai/Qwen3.8-Flash-Next-DFlash2
   It's about 20% faster than MTP in their table when concurrency 1.
2. Oct 19, 2026 · Opinion / analysis · 1 source
   Don't sleep on Nemotron 3.5 Super VL
   Nemotron 3.5 Super VL 120B-A12B: Hallucionated salt for citizen to merchant, left out church for settlers, left out required landmarks for merchant and aristocrat
3. Oct 19, 2026 · Open-source release · 1 source
   sgl-project/sglang v0.5.22
   | Model | Type | Cookbook |
4. Oct 19, 2026 · Opinion / analysis · 1 source
   Qwen 3.8 Flash Next optimized for two CMP 170hx GPUs
   I've been working on optimizing Qwen 3.8 Flash Next on my local setup with two CMP 170HX cards and finally got to a place where I feel like the results are worth sharing.
5. Oct 18, 2026 · Opinion / analysis · 1 source
   Little Sweet Room Challenge | Strata Fork | Qwen 3.8 FN NVFP4 | DeepSeek Harness
   Another little challenge I seen on a DeepSeek Thread.
6. Oct 18, 2026 · Opinion / analysis · 1 source
   You don’t need that expensive of a build for local llms.
   Been seeing a lot of people with their 64gb of ddr5 4x3090 or whatever strata builds and it’s been sorta depressing knowing i cant afford that but i finally finished my setup today so here we go,
7. Oct 18, 2026 · Opinion / analysis · 1 source
   Nine models, five coding tasks, one 16GB GPU
   I ran five coding tasks against nine models on one RX 9060 XT (16 GB).
8. Oct 17, 2026 · Opinion / analysis · 1 source
   2 gaming PCs + an M4 Pro MacBook over Ethernet + 2 iPhone 17 Pro Maxes over USB-C: Qwen3.8 Flash Next at 37-39 tok/s at 60k context (262k layout in the works)
   Qwen 3.8 Flash Next is a 48-layer mixture-of-experts model with 512 experts per layer, and even at 3-bit it’s far bigger than my 24 GB MacBook.
9. Oct 17, 2026 · Opinion / analysis · 1 source
   32GB to 64GB of RAM took Qwen3.8-Flash-Next from IQ2_XS (2-bit) to Unsloth UD-IQ4_XS (4-bit) on a single 4090
   I went from 32GB to 64GB of DDR5-5800 this week and moved my Strata setup off IQ2XS.
10. Oct 17, 2026 · Opinion / analysis · 1 source
    Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming
    I have been working on implementing adaptive KV streaming and RDNA 4 / ROCm optimizations for llama.cpp to enable running Qwen3.8-27B (IQ4XS weights, Q80/Q40 KV cache) with its full 262,144-token context window on a single 16 GB RX 9070 XT GPU at 80+ t/s decode.
11. Oct 17, 2026 · Open-source release · 2 sources
    StepFun Releases Step 5 Preview Open Weights: 600B MoE, 27B Active, 1M Context
    StepFun has released Step 5 Preview as open weights.
12. Oct 17, 2026 · Opinion / analysis · 1 source
    Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF
    Defiance puts DavidAU's Turbo-Brilliance "AI Construct" modes on HauhauCS's uncensored Qwen3.8-27B.
13. Oct 17, 2026 · Opinion / analysis · 1 source
    llama.cpp has come a long way
    I started out back in March 2025 running Mistral Nemo Q4KM with 8K F16 on an AMD Radeon RX 7800 XT 16GB, with first ollama and then lmstudio.
14. Oct 15, 2026 · Tutorial / explainer · 1 source
    How do you publish on arXiv without an institution? Looking for an endorser or reviewer for a paper on modular, toolchain-validated small code models [R]
    I wanted to publish my very first scientific article on platforms like arXiV or ResearchGate, but because I do not have the scientific community around me that can help me with that - I need to ask here how you'd overcome this.
15. Oct 15, 2026 · Opinion / analysis · 1 source
    Ternary Bonsai 2 Family: 2B, 4B, & 9B based on Qwen 3.5
    Going to try these out to see if they have the same looping behaviors that other Bonsai models have had.
16. Oct 15, 2026 · Model release · 5 sources
    NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
    NVIDIA Nemotron 3.5 Super VL is an open, customizable multimodal model for these specialized workflows.
17. Oct 15, 2026 · Open-source release · 1 source
    unslothai/unsloth v0.1.908-beta: NVIDIA Nemotron 3.5 + Meta Muse Spark Open
    Run NVIDIA's new Nemotron 3.5 models and Meta's new Muse Spark Open locally in Unsloth.
18. Oct 15, 2026 · Opinion / analysis · 1 source
    What a £300, 35W Snapdragon mini PC can actually do 👀: 35B coder at ~30 tok/s, and an 80B model that doesnt even fit in RAM running at 17 tok/s
    £299.99 Lenovo IdeaCentre Mini (Snapdragon X1-26-100, 32GB, CPU only):
19. Oct 15, 2026 · Open-source release · 1 source
    ggml-org/llama.cpp v0.7.0
    The server gets a refactored chat session API (commonchatsession, part-based formatted output), much better slot and prompt-cache restore behavior, decision-model support for the new architectures and a new default port (9931), the web UI gains a full models manager, and ggml is updated to v0.27.0.
20. Oct 15, 2026 · Research paper · 1 source
    HyCo: Hybrid Drafting for Cost-Efficient Mobile-Cloud Speculative Decoding
    Device-cloud speculative decoding reduces server computation, but growing a deep draft tree with a small language model (SLM) requires repeated passes on a mobile neural processing unit (NPU).
21. Oct 15, 2026 · Opinion / analysis · 1 source
    4x R9700 builds with $127 used EPYCs: $10.7k all-in per machine. I built three (384GB VRAM)
    I saw the 4x R9700 build for a 10-person startup here a few days ago and wanted to share my results, since I opted to spend most of the money on the cards.
22. Oct 14, 2026 · Open-source release · 4 sources
    Fish Audio Drama 3 Technical Report
    We introduce Fish Audio Drama 3, an open-source speech generation model that turns text into directed performances, built on a 35B-parameter mixture-of-experts backbone that activates 3B parameters per token.
23. Oct 14, 2026 · Research paper · 1 source
    Long-Context Drafting Does Not Require Long-Context Drafters
    Speculative decoding pairs a cheap drafter with parallel verification against the target.
24. Oct 14, 2026 · Product / feature launch · 1 source
    New model from Telperion: Qwen3.8-27B-MLA & Qwen3.8-27B-GLA-g2. 4x smaller KV cache, smarter, and more efficient reasoning.
    Today, we're announcing the release of our latent attention variants of Qwen3.8-27B.
25. Oct 14, 2026 · Open-source release · 1 source
    NVIDIA/TensorRT-LLM v1.4.0rc0
    DeepSeek MLA models on SM120/SM121 GPUs (for example RTX PRO 6000 Blackwell) may crash with an illegal memory access when CUDA graphs are captured with BF16 BMM.
26. Oct 13, 2026 · Open-source release · 1 source
    Comfy-Org/ComfyUI v0.40.0
27. Oct 13, 2026 · Opinion / analysis · 1 source
    Instead of waiting for an M5 Ultra 512 GB: pairing a new M5 Ultra 256 GB with my M3 Ultra 512 GB for disaggregated prefill/decode (GLM-5.3-Flash, MLX, Thunderbolt 5 RDMA)
    The problem I'm solving: I have an M3 Ultra 512 GB and the memory is still the reason to own it, but prefill on long prompts is slow.
28. Oct 13, 2026 · Opinion / analysis · 1 source
    PSA for 3090 FE owners: Running LLMs overheats your VRAM
    Since LLM token generation maxes out VRAM's read capacity, I checked VRAM temps with Nvidia's stock fan/power curves and discovered that VRAM hits 112C within seconds of starting token generation.
29. Oct 13, 2026 · Opinion / analysis · 1 source
    Horsey challenge | Strata Fork | Qwen 3.8 FN NVFP4
    I seen a post earlier and seen i believe it was a iq3 i think and it was pretty dope so i thought i try out the NVFP4.
30. Oct 12, 2026 · Opinion / analysis · 1 source
    Running unsloth/Qwen3.8-27B-UD-IQ4_XS 262k q4_0 KV + vision on 16GB VRAM @ 80t/s and 1500pp
    I get a lot of hate whenever I tell people I run a Q4 quant of Qwen3.8 27B with 262k KV 80t/s with vision, all while running 2 screens on windows on the same card.

Often appears with