E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks
We propose E8-QAT, a 2-bit quantization-aware training framework that uses the $E8$ lattice as an implicit eight-dimensional codebook.
ProofPaper ↗
Key points
- Low-bit quantization reduces the memory footprint of large language models, but preserving accuracy with efficient training remains challenging.
- Scalar quantization encodes weights independently, while explicit-codebook vector quantization introduces costly search during training.
- It jointly quantizes weight groups through arithmetic projection without storing or searching an explicit codebook during the training forward pass.
- A 2-bit core encoding and sparse extension preserve the selected lattice points exactly, adding less than 0.1% to core index storage.
Sources (1)
- [1]E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice CodebooksarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 06:17 AM
We propose E8-QAT, a 2-bit quantization-aware training framework that uses the $E_8$ lattice as an implicit eight-dimensional codebook.
Low-bit quantization reduces the memory footprint of large language models, but preserving accuracy with efficient training remains challenging.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 19, 2026MIRA: A Multimodal Illustrated Deep Research Agent for Controllable Reports
- Oct 19, 2026CVaR-RMD: CVaR Selection Pressure for Robust Metaheuristic Discovery
- Oct 19, 2026LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models
- Oct 19, 2026Vector-Quantized World Model: Latent Feedback Motion Planning for Robot Manipulation
- Oct 18, 2026Outcome-Anchored Credit Reallocation for Agentic Knowledge Graph Question Answering