Ternary Bonsai 2 Family: 2B, 4B, & 9B based on Qwen 3.5
Going to try these out to see if they have the same looping behaviors that other Bonsai models have had.
Proof1 community thread
Key points
- Still requires their fork of llama.cpp (branch prism) so dont expect it to work with vanilla llama.cpp.
- Edit: You can use with regular llama.cpp!
- Will see if there is some way to support this for AnythingLLM Mobile via LLama.rn fork since if anything this will hopefully beat out regular Qwen 3.5 1.7B or 0.8B.
- Can test on iOS today with LocallyAI on latest version.
Sources (1)
- [1]Ternary Bonsai 2 Family: 2B, 4B, & 9B based on Qwen 3.5r/LocalLLaMA (top, daily) · Oct 15, 09:45 PM
Going to try these out to see if they have the same looping behaviors that other Bonsai models have had.
Still requires their fork of llama.cpp (branch prism) so dont expect it to work with vanilla llama.cpp.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint
- Oct 15, 2026What a £300, 35W Snapdragon mini PC can actually do 👀: 35B coder at ~30 tok/s, and an 80B model that doesnt even fit in RAM running at 17 tok/s
- Oct 15, 2026ggml-org/llama.cpp v0.7.0
- Oct 15, 2026NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
- Oct 15, 2026Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio
- Oct 14, 2026GPT-Image-style photo editing on your own GPU (12 GB+): I built a zero-setup app for Qwen-Image 2.1 on stable-diffusion.cpp
