Budget upgrade for a 31B model? Dual GPUs over PCIe x1 or another approach?
Looking for advice on upgrading my AI server on a budget.
Proof1 community thread
Key points
- My goal is to run Gemma 4 31B Q5KM (~23GB) at 16K+ context (q8), achieving at least 15 t/s.
- Currently running the 31B Q5 model at 53% CPU / 47% GPU offloading, which is somewhat useable but slow (in 243, out 2.3 t/s (mtp 2.7)).
- Add an RTX 5060 Ti 16GB (~$750 new here in Thailand, no used available), move the 3060 to a powered PCIe 3.0 x1 riser, and use layer splitting.
- --> 2× Tesla P100 16GB (or similar) on a cheap X79/X99 Xeon 32-64gb ddr3 system (~$600–900 total).
Sources (1)
- [1]Budget upgrade for a 31B model? Dual GPUs over PCIe x1 or another approach?r/LocalLLaMA (top, daily) · Oct 20, 02:50 AM
Looking for advice on upgrading my AI server on a budget.
My goal is to run Gemma 4 31B Q5_K_M (~23GB) at 16K+ context (q8), achieving at least 15 t/s.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026jax-ml/jax jax-v0.12.0: JAX v0.12.0
- Oct 19, 2026NVIDIA Open Sources Run:ai Under Apache 2.0 for Kubernetes GPU Orchestration
- Oct 18, 202672 hours, four big open weights drops. The only one that fits in 24GB came from NVIDIA, of all companies.
- Oct 18, 2026Agens Volundr 32B v1: we fixed what you flagged in the Preview. 0/50 loops, session continuation, and an MIT engine that runs it with 262K context (measured to 239K) within a 16 GB budget
- Oct 18, 2026My 5070 ti/3070 Chimera build.
- Oct 17, 2026Fish Audio Drama 3 says it needs a 48 GB NVIDIA card. My 16 GB Radeon disagrees.