Detecting Unreliable Classifications from Quantized Models
We show a label-free alternative: comparing a model's own output before and after quantization.
ProofPaper ↗
Key points
- Post-training INT8 quantization is typically validated by checking accuracy retention, which requires ground-truth labels often unavailable in low-resource clinical settings.
- We compute cosine similarity between the FP32 and INT8 models' output probability vectors for each prediction, which requires only a single forward pass.
- This output-agreement similarity separates lower-accuracy predictions across four evaluated architectures (ResNet-50, VGG-16-BN, Wide ResNet-50, and ViT-B/16) under fixed-percentile splits, outperforming a confidence-only baseline for all four.
- These results show that quantized models can be compared to their baselines to assess their drift and determine whether they are well suited for clinical deployment.
Sources (1)
- [1]Detecting Unreliable Classifications from Quantized ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 03:41 PM
We show a label-free alternative: comparing a model's own output before and after quantization.
Post-training INT8 quantization is typically validated by checking accuracy retention, which requires ground-truth labels often unavailable in low-resource clinical settings.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026openbmb/MiniCPM-V-4.7-1B
- Oct 12, 2026openbmb/MiniCPM-V-4.7-35B-A3B
- Oct 12, 2026Deploy decision models with llama.cpp on Amazon SageMaker AI
- Oct 12, 2026Compressed a RoBERTa AI-text detector from ~500 MB to ~97 MB so it runs client-side in the browser, plus a 26 MB image detector [P]
- Oct 12, 2026StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions
- Oct 12, 2026Rethinking LLM-Guided Visual Question Answering: Does Deeper Knowledge Integration Really Help?