I clustered two old Android phones to run a 27B LLM locally. Bypassed RAM limits via C++ RPC split and Hadamard Ternary math (No overheating)

Key points

• I wrote a custom C++ engine (fork of llama.rn) to cluster two Android phones over a simple USB cable.1
• Model: Ternary-Bonsai-2-27B (compressed to 2.06 bpw / 6.52 GiB).1
• Cluster Mode (Xiaomi redme 100 pro max + ROG 7): Layers are split 4.36 GiB / 2.15 GiB over the USB cable via custom RPC.1
• Speed: ~2.8 tok/s (bottlenecked by the slower phone, but perfectly synced).1

Sources (1)

• [1]I clustered two old Android phones to run a 27B LLM locally. Bypassed RAM limits via C++ RPC split and Hadamard Ternary math (No overheating)
  r/LocalLLaMA (top, daily) · Oct 13, 04:00 AM

  I saw a lot of posts here struggling to run 27B models on 8GB VRAM or complaining about phones turning into space heaters.

  I wrote a custom C++ engine (fork of llama.rn) to cluster two Android phones over a simple USB cable.

Extractive summary: sentences quoted from the sources.