I tested Qwen3.8 27B KLD for 21 Models at 9 different KV quants that fit on 16GB VRAM (189 benchmarks).
I calculated KLD and perplexity for Qwen3.8 27B models that fit 16GB VRAM:
Proof1 community thread
Key points
- There is very little difference between q80/q40 and q40/q80
- Going up a model quant and running at q40/q40 is almost always better than running the quant below at f16/f16 in terms of KLD.
- If you are going from q6->q8 model quant it might be bigger gains increasing KV quant, due to diminishing gains going up model quants.
- Bonsai and swift quants are included but it does not really make sense to calculate KLD since they are finetunes and the KLD will differ significantly.
Sources (1)
- [1]I tested Qwen3.8 27B KLD for 21 Models at 9 different KV quants that fit on 16GB VRAM (189 benchmarks).r/LocalLLaMA (top, daily) · Oct 15, 08:11 AM
I calculated KLD and perplexity for Qwen3.8 27B models that fit 16GB VRAM:
There is very little difference between q8_0/q4_0 and q4_0/q8_0
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026Calibration Language Shapes Block-Influence Pruning in Multilingual Language Models
- Oct 14, 2026Tessera-27B-B-CJK, Zellige AIs new finetune of Qwen with novel approaches to how we think about token savings
- Oct 14, 2026New model from Telperion: Qwen3.8-27B-MLA & Qwen3.8-27B-GLA-g2. 4x smaller KV cache, smarter, and more efficient reasoning.
- Oct 14, 2026qwen3.8-flash-next-iq3_s vs -27B-GGUF:UD-Q4_K_XL
- Oct 14, 2026cuda, hip: GDN chunked kernel by am17an · Pull Request #29353 · ggml-org/llama.cpp
- Oct 13, 2026SuperLocalMemory
