ACE-Compression: An Ability-Centric Evaluation of Compressed Large Language Models
Model compression makes large language models deployable, but today's evaluation tells an incomplete story.
ProofPaper ↗
Key points
- This misses a critical fact: compression perturbs the distributed knowledge embedded across billions of weights, and task scores alone cannot reveal which underlying abilities survive and which quietly collapse.
- To solve this question, we introduce ACE-Compression, an ability-centric evaluation framework.
- Our framework probes four deployment-facing dimensions: procedural reasoning and constraint execution, knowledge grounding and truthfulness, social cognition and affective understanding, and persona fidelity and stylistic consistency.
- Across more than 35 compressed checkpoints from six open-weight LLMs-covering quantization and pruning-we find that compression redistributes ability profiles.
Sources (1)
- [1]ACE-Compression: An Ability-Centric Evaluation of Compressed Large Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 09:26 AM
Model compression makes large language models deployable, but today's evaluation tells an incomplete story.
This misses a critical fact: compression perturbs the distributed knowledge embedded across billions of weights, and task scores alone cannot reveal which underlying abilities survive and which quietly collapse.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026A Unified Attack Framework against Statistical Generative Watermarks for Large Language Models
- Oct 19, 2026Introducing Clairic, a new System One class of extraction models
- Oct 19, 2026From Black Box to Controller: Inference-Time Steering of LLM via Feature Amplification
- Oct 19, 2026LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models
- Oct 19, 2026MapSteal: Map-based Autonomous Knowledge Stealing Attack Against Large Language Models
- Oct 18, 2026Where and Why Search-Augmented LLMs Fail When Reasoning over Rapidly Evolving Knowledge