LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models
To address this, we propose Locally Isolated Rotation Quantization (LIRQuant), a sensitivity-aware framework that selectively applies transformations.
ProofPaper ↗
Key points
- Post-Training Quantization (PTQ) enables efficient deployment of Large Language Models (LLMs), yet activation outliers remain a key challenge for low-bit quantization.
- Existing global transformation methods overlook heterogeneous channel sensitivity to quantization noise, leading to suboptimal accuracy-efficiency trade-offs.
- LIRQuant identifies sensitive channels using a noise-weighted sensitivity metric, applying expressive learnable rotations only to these channels and lightweight fixed rotations to the remaining ones.
- Meanwhile, LIRQuant calibrates LLaMA-3-8B in only 25 minutes with 20.3 GB peak GPU memory, achieving the fastest calibration and the lowest memory footprint among recent learnable-rotation methods, and its fused inference kernels deliver up to $2.3\times$ prefill speedup and $3.5\times$ memory saving over FP16.
Sources (1)
- [1]LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 02:24 AM
To address this, we propose Locally Isolated Rotation Quantization (LIRQuant), a sensitivity-aware framework that selectively applies transformations.
Post-Training Quantization (PTQ) enables efficient deployment of Large Language Models (LLMs), yet activation outliers remain a key challenge for low-bit quantization.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 18, 2026MOSAIC: Multi-Objective Search for Allocating Inference Cache Across LLM Decoder Layers
- Oct 18, 2026Do LLMs Reproduce the Age Gap in Loneliness? Evaluating Survey Simulation against European Survey Data
- Oct 17, 2026First-Order LoRA Training with Phone NPUs
- Oct 17, 2026Medical Fine-Tuning Reshapes, Not Reduces, Patient-Facing AI Risk: A Rubric-Based, LLM-Judge Evaluation Framework
- Oct 17, 2026llama.cpp has come a long way
- Oct 16, 2026NL2SQL with Iterative Self-Refinement