Update: Converting dense models into Mixture-of-Experts
I have also released GGUF versions for both the 0.5B and the 1.5B models which can be found here bayliner1980/Qwen2.5-0.5B-MoE-A0.3B-GGUF and here bayliner1980/Qwen2.5-1.5B-MoE-A0.9B-GGUF.
Proof1 community thread
Key points
- Since my last post I have continued working on the conversion process and also converting new models.
- My latest model is the Qwen2.5-1.5B-MoE-A0.9B.
- WikiText-2 test perplexity (1024-token windows) 10.41 14.63 (+41%)
- This is the first conversion that's clearly faster than the dense model on GPU too.
Sources (1)
- [1]Update: Converting dense models into Mixture-of-Expertsr/LocalLLaMA (top, daily) · Oct 13, 11:50 AM
I have also released GGUF versions for both the 0.5B and the 1.5B models which can be found here bayliner1980/Qwen2.5-0.5B-MoE-A0.3B-GGUF and here bayliner1980/Qwen2.5-1.5B-MoE-A0.9B-GGUF.
Since my last post I have continued working on the conversion process and also converting new models.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026SonicBoom: 52 tok/s decode on a laptop RTX 3050 (llama.cpp: 45) with the same quantized kernels, the difference is a graph compiler. No Python.
- Oct 13, 2026Running decision models: Picking quant, VRAM usage
- Oct 13, 2026I clustered two old Android phones to run a 27B LLM locally. Bypassed RAM limits via C++ RPC split and Hadamard Ternary math (No overheating)
- Oct 13, 2026Fish Audio Drama 3 Technical Report
- Oct 12, 2026openbmb/MiniCPM-V-4.7-35B-A3B
- Oct 12, 2026strix-llama new branch: prompt processing +21–54% over Vulkan and up to 2.7× over stock ROCm kernel at 128k, same outputs quality
