ResearchResearch paperEfficiency & Inference · Large Language Models1 source · Oct 10, 2026

FidelityQ: Preserving Predictive Fidelity in Ultra-Low-Bit Language Model Quantization

We propose FidelityQ, a scalar quantization framework that preserves predictive fidelity at weight precisions down to 2 bits.

Ultra-low-bit scalar quantization offers a regular arithmetic structure for efficient deployment of large language models, but preserving predictive behavior at extreme precision remains challenging: with only a few reconstruction levels, accurate weight representation must translate into consistent predictions from a small calibration set.
For accurate quantization, we introduce Information-Aligned Analytic Coding (IAC), which shapes each weight row toward a centered symmetric source via randomized Hadamard mixing and fits a mirrored four-level minimum-distortion quantizer to its empirical weights.
Our analysis further shows that residual predictive discrepancies are heavy-tailed and concentrate on tokens the teacher predicts confidently, motivating Knowledge-Intensive Calibration (KIC) for consistent quantization.
Experiments on the LLaMA and Qwen families demonstrate the effectiveness of FidelityQ in weight-only and joint quantization settings.

[1]FidelityQ: Preserving Predictive Fidelity in Ultra-Low-Bit Language Model Quantization
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 02:06 PM
We propose FidelityQ, a scalar quantization framework that preserves predictive fidelity at weight precisions down to 2 bits.
Ultra-low-bit scalar quantization offers a regular arithmetic structure for efficient deployment of large language models, but preserving predictive behavior at extreme precision remains challenging: with only a few reconstruction levels, accurate weight representation must translate into consistent predictions from a small calibration set.

Extractive summary: sentences quoted from the sources.

Before this