CUDA: faster MMVQ for low-bit types by am17an · Pull Request #30517 · ggml-org/llama.cpp
big speedup, but only for specific quantizations like Q2K and Q3K
Proof1 community thread
Sources (1)
- [1]CUDA: faster MMVQ for low-bit types by am17an · Pull Request #30517 · ggml-org/llama.cppr/LocalLLaMA (top, daily) · Oct 17, 02:33 PM
big speedup, but only for specific quantizations like Q2_K and Q3_K
Extractive summary: sentences quoted from the sources.
Before this
- Oct 17, 2026Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming
- Oct 17, 2026model : add Kolibri-1 (Aleph-Alpha) support by riteme · Pull Request #30673 · ggml-org/llama.cpp
- Oct 17, 2026Nemotron 3.5 Bolt, Muse Spark Open, and a Fast LFM2.5 24B
- Oct 15, 2026Ternary Bonsai 2 Family: 2B, 4B, & 9B based on Qwen 3.5
- Oct 15, 2026QUASAR 4-bit QAT checkpoints for Qwen3.5, Gemma 4 and Muse Glimmer: closer to BF16 than Google’s and NVIDIA’s 4-bit releases
- Oct 15, 2026Welcome FLUX.3 Image Klein in Diffusers
