capy-flash v0.7.1: Qwen3.8-Flash-Next (125B MoE) now runs in 64 GB of RAM (234 tok/s on two 16 GB cards), and Qwen3.8-27B 3-bit fits on a single 16 GB card (174 tok/s)

Key points

• Until now it needed my setup (two 16 GB cards and 96 GB of RAM) or better. v0.7 is about smaller machines: start with what you have, add a second card later, same engine and same model files.1
• Flash-Next on 64 GB of RAM, one or two GPUs.1
• Qwen3.8-27B on one 16 GB card, 3-bit GSQ weights (ISTA-DASLab), 64k context, vision, MTP.1
• If you have a single 16 GB card, Strata is probably already the better choice for Flash-Next today. capy-flash is at its best with two cards, where it adds tensor parallelism and a VRAM expert cache shared by both GPUs.1

Sources (1)

• [1]capy-flash v0.7.1: Qwen3.8-Flash-Next (125B MoE) now runs in 64 GB of RAM (234 tok/s on two 16 GB cards), and Qwen3.8-27B 3-bit fits on a single 16 GB card (174 tok/s)
  r/LocalLLaMA (top, daily) · Oct 22, 01:34 PM

  I've been building capy-flash, an open-source inference engine for running Qwen3.8-Flash-Next (125B MoE) and Qwen3.8-27B on consumer RTX 50 cards.

  Until now it needed my setup (two 16 GB cards and 96 GB of RAM) or better. v0.7 is about smaller machines: start with what you have, add a second card later, same engine and same model files.

Extractive summary: sentences quoted from the sources.