Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~40-50 tok/sec decode & 1300-1500 tok/sec prefill at 131k ctx on 12GB VRAM + 32GB RAM + NVME

Key points

• Don’t see IQ3S and instantly dismiss it, this recipe uses an abliterated version of ISTADASLab’s GSQ-RCO Qwen3.8-FN, which retains ~99% of BF16.1
• Obviously there are rough edges with any quantization method but in comparison to all generic imatrix quants, this is the best quant for this model across my testing (for this hardware tier - higher precision uniform quants will perform better obviously).1
• And to have it running at these speeds on only ~44GB combined VRAM/RAM with an SSD is kind of ridiculous.1
• This fork has tonnes of architectural changes, all are very experimental.1

Sources (1)

• [1]Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~40-50 tok/sec decode & 1300-1500 tok/sec prefill at 131k ctx on 12GB VRAM + 32GB RAM + NVME
  r/LocalLLaMA (top, daily) · Oct 16, 09:18 PM

  I got Qwen3.8-Flash-Next-GSQ-RCO-Abliterated running at IQ3_S with just 12GB VRAM + 32GB RAM + NVME, achieving 40-50 tok/sec decode & 1300-1500 tok/sec prefill @ 131k context (44k thinking budget), on a custom fork of Strata.

  Don’t see IQ3_S and instantly dismiss it, this recipe uses an abliterated version of ISTADASLab’s GSQ-RCO Qwen3.8-FN, which retains ~99% of BF16.

Extractive summary: sentences quoted from the sources.