What actually breaks agents on a 4B model? Notes from 50 runs on Qwen3.5-4B
Agents: Pi 1.3.0, goose 1.55.0, Crush 0.98.1, Cline 3.0.73, OpenClaw 2026.10.5 (in its --local-model-lean mode), OpenCode 1.19.3, Hermes 0.22.2, Qwen Code 0.25.3, Codex CLI 0.163.0 and Atomic Agent 0.6.9.
Proof1 community thread
Key points
- I ran 10 agent harnesses on the same local model and the same 5 small tasks, mostly to see where a 4B actually falls over.
- Setup: Qwen3.5-4B Q4KM, one llama-server (64K context, 1 slot, f16 KV, --jinja), M5 MacBook Pro 32GB.
- I expected malformed tool calls to be the main failure on a 4B.
- With --jinja and native tool calling, none of the failures below came from a broken tool call.
Sources (1)
- [1]What actually breaks agents on a 4B model? Notes from 50 runs on Qwen3.5-4Br/LocalLLaMA (top, daily) · Oct 21, 09:42 PM
Agents: Pi 1.3.0, goose 1.55.0, Crush 0.98.1, Cline 3.0.73, OpenClaw 2026.10.5 (in its --local-model-lean mode), OpenCode 1.19.3, Hermes 0.22.2, Qwen Code 0.25.3, Codex CLI 0.163.0 and Atomic Agent 0.6.9.
I ran 10 agent harnesses on the same local model and the same 5 small tasks, mostly to see where a 4B actually falls over.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 21, 2026Recursive launches Flint, a laptop coding model it says was largely built by AI
- Oct 21, 2026I built a small on-device voice front-end for local assistants: type any wake word, keep only your voice before STT (5 models, 7.5 MB, Apache-2.0, early preview)
- Oct 20, 2026Qwen3.8-Flash-Next vs Opus 5.5: Six-way benchmark on a real Rust feature with Claude Code and Pi
- Oct 20, 2026How I built my ‘everything’ app
- Oct 20, 2026Voxtral 3
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
