capy-flash: Qwen3.8-Flash-Next (125B MoE) in full NVFP4 on two 16 GB consumer GPUs
capy-flash is the evolution of my ninfer-tp2 engine.
Proof1 community thread
Key points
- It still runs Qwen3.8-27B as before, and now also Qwen3.8-Flash-Next 125B-A6B with the experts offloaded to RAM.
- It uses NVIDIA's official NVFP4 checkpoint as-is and runs both GPUs in tensor parallel.
- On my machine (2× RTX 5070 Ti 16 GB, Ryzen 9 9900X, 96 GB DDR5), with the same NVFP4 weights, it does ~100 tok/s generation vs 78 for Strata, and ~3.4k tok/s prefill on a 15k prompt vs 2.7k.
- Where the two halves cancel, the error blows up: on Qwen3 at TP=2, large errors happen ~5× as often as on one GPU.
Sources (1)
- [1]capy-flash: Qwen3.8-Flash-Next (125B MoE) in full NVFP4 on two 16 GB consumer GPUsr/LocalLLaMA (top, daily) · Oct 15, 08:20 PM
capy-flash is the evolution of my ninfer-tp2 engine.
It still runs Qwen3.8-27B as before, and now also Qwen3.8-Flash-Next 125B-A6B with the experts offloaded to RAM.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026QUASAR 4-bit QAT checkpoints for Qwen3.5, Gemma 4 and Muse Glimmer: closer to BF16 than Google’s and NVIDIA’s 4-bit releases
- Oct 15, 2026huggingface/transformers v5.20.0: Release v5.20.0
- Oct 15, 2026What a £300, 35W Snapdragon mini PC can actually do 👀: 35B coder at ~30 tok/s, and an 80B model that doesnt even fit in RAM running at 17 tok/s
- Oct 15, 2026NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
- Oct 14, 2026GPT-Image-style photo editing on your own GPU (12 GB+): I built a zero-setup app for Qwen-Image 2.1 on stable-diffusion.cpp
- Oct 14, 2026mistralai/Timestral-209M
