DPA-MoE: A Runtime Precision Control Framework for MoE Inference

Key points

• Mixture-of-Experts (MoE) models reduce computation by routing each token to a small set of experts, but each selected expert is still used at a fixed precision.1
• DPA-MoE encodes expert feed-forward weights as prefix-decodable chunks using SharedScale AnyBCQ, profiles per-expert reconstruction residuals offline, and allocates chunks at inference time using live router gate scores.1
• This post-routing allocation lets a single encoded MoE checkpoint operate at multiple effective weight budgets without retraining or re-encoding.1
• Experiments on Qwen3-30B-A3B and DeepSeekMoE-16B show that DPA-MoE improves KL divergence and downstream accuracy over uniform chunk allocation at matched budgets, with modest steady-state throughput overhead in our prototype.1

Sources (1)

• [1]DPA-MoE: A Runtime Precision Control Framework for MoE Inference
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 06:58 AM

  We present DPA-MoE, a runtime precision control framework that makes routed experts any-precision.

  Mixture-of-Experts (MoE) models reduce computation by routing each token to a small set of experts, but each selected expert is still used at a fixed precision.

Extractive summary: sentences quoted from the sources.