Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCR
Character Error Rate (CER), the standard metric for OCR and vision-language models (VLMs), charges a typo as much as a wrong name.
ProofPaper ↗
Key points
- We introduce semCER, which keeps CER's alignment and denominator and charges each grapheme edit a severity from an LLM judge, with no word segmentation.
- Normalized CER keeps the score of a whole transcription, where it tracks readers as well as semCER or better, and semCER breaks that score down into named edits.
- On pairs drawn where CER and semCER disagreed, readers side with semCER more often than with normalized CER in English and Chinese, and equally often in Vietnamese.
- Across 16 languages, the judge charges a wrong diacritic or tone mark almost nothing in Spanish but nearly a wrong letter's severity in Telugu.
Sources (2)
- [1]Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCRarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 04:44 PM
Character Error Rate (CER), the standard metric for OCR and vision-language models (VLMs), charges a typo as much as a wrong name.
We introduce semCER, which keeps CER's alignment and denominator and charges each grapheme edit a severity from an LLM judge, with no word segmentation.
- [2]Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCRHugging Face Daily Papers · Oct 14, 12:00 AM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
- Oct 13, 2026Gemini 4 Argon: Built to solve your hardest problems
- Oct 13, 2026LongForcing: Scaling Temporal Manifold Matching for Long-Horizon World Rollout
- Oct 13, 2026LinMorph: Learning to Morph Weights for Efficient Video DiT Linearization
- Oct 13, 2026Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation
- Oct 13, 2026GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning
