Strata with multi gpu explanation
I am mostly familiar with llama.cpp but have been using QFN - UD-Q4KXL with Strata and need some help understanding how it works with multiple gpu's.
Proof1 community thread
Key points
- I initially started with a single gpu and got about 70 t/s.
- I expected RAM usage to come down since all 3 gpu's VRAM was filled so I don't quite understand how Strata works here.
- Is it using the other gpu's for context or what?
- Edit the strata json file and add this argument, where 50 is your ram budget in gb.
Sources (1)
- [1]Strata with multi gpu explanationr/LocalLLaMA (top, daily) · Oct 19, 12:55 PM
I am mostly familiar with llama.cpp but have been using QFN - UD-Q4_K_XL with Strata and need some help understanding how it works with multiple gpu's.
I initially started with a single gpu and got about 70 t/s.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Calling all GPUs: help us benchmark SonicBoom v0.0.3 — a compiler-first AI runtime that beats llama.cpp on decode
- Oct 18, 2026Agens Volundr 32B v1: we fixed what you flagged in the Preview. 0/50 loops, session continuation, and an MIT engine that runs it with 262K context (measured to 239K) within a 16 GB budget
- Oct 17, 2026Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming
- Oct 16, 2026cpu : reduce AVX2 tiled accumulators together by trivalik · Pull Request #30296 · ggml-org/llama.cpp
- Oct 15, 2026What a £300, 35W Snapdragon mini PC can actually do 👀: 35B coder at ~30 tok/s, and an 80B model that doesnt even fit in RAM running at 17 tok/s
- Oct 15, 2026NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning