DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving
Serving a Mixture-of-Experts (MoE) model pays for expert precision twice: the resident footprint pays for every expert the router can reach, while each batch-one decode step pays only for the experts it selects.
ProofPaper ↗
Key points
- These are two different rate functionals over the same per-expert widths, yet published per-expert MoE allocators constrain only the first.
- We call the difference the store-read gap; in the released BitsMoE checkpoint, 2.54 bits per weight (bpw) of storage become 7.50 bpw of decode traffic.
- We propose DuoRate, which stores once and reads per workload.
- On Qwen3-30B-A3B, DuoRate reading 2.3 bpw exceeds every 2-bit-class baseline by 6.3 to 47.0 points of average accuracy, and at 2.9 bpw it matches 4-bit formats that read 1.4-1.6 bpw more.
Sources (1)
- [1]DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts ServingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 10:07 AM
Serving a Mixture-of-Experts (MoE) model pays for expert precision twice: the resident footprint pays for every expert the router can reach, while each batch-one decode step pays only for the experts it selects.
These are two different rate functionals over the same per-expert widths, yet published per-expert MoE allocators constrain only the first.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash
- Oct 9, 2026The Thinking Leakage in Reinforcement Learning for LLMs
- Oct 9, 2026Multi-Stream Attention Residuals
- Oct 8, 2026STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules
- Oct 8, 2026MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement
- Oct 8, 2026One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts