When Dynamic Scaling Fails: A Scale-State Feedback Governs Recurrent-State Quantization in Linear-Attention and SSM LLMs
Linear-attention and state-space LLMs carry long context in a fixed-size recurrent state, and quantizing that state is the natural way to cut its cost during long decode.
ProofPaper ↗
Key points
- Unlike a weight tensor, the state is rewritten at every step, so a quantizer applied at each step sits inside the recurrence and its errors compound.
- INT4 that leaves synthetic recall perfectly intact at the start destroys it a thousand steps later.
- The mechanism is a feedback between the quantizer scale and the state it is derived from.
- Allocating bits across layers does help, but under article-disjoint splits a learned allocation collapses onto the trivial rule of protecting the first few layers.
Sources (1)
- [1]When Dynamic Scaling Fails: A Scale-State Feedback Governs Recurrent-State Quantization in Linear-Attention and SSM LLMsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 07:30 PM
Linear-attention and state-space LLMs carry long context in a fixed-size recurrent state, and quantizing that state is the natural way to cut its cost during long decode.
Unlike a weight tensor, the state is rewritten at every step, so a quantizer applied at each step sits inside the recurrence and its errors compound.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 10, 2026Toward System One Clinical Intelligence: Parameter-Efficient Fine-Tuning and Counterfactual Explainability of Decision Foundation Models
- Oct 10, 2026SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usability
- Oct 9, 2026Microsoft's Decision-1 model enters the fast-growing AI decision model race
- Oct 9, 2026ReplaySSM: Cache SSM Inputs, Not State
- Oct 9, 2026DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
- Oct 9, 2026RAMP: Recurrent Adaptive Mixed Precision for Efficient Prefill in Recurrent-Depth Language Models