When Models Misread Ink: Auditing VLMs for Arabic Calligraphy Recognition
Arabic calligraphy is a high-stakes cultural visual domain for Muslim communities, yet VLM evaluations rarely test its fine-grained script traditions.
ProofPaper ↗
Key points
- We audit 12 VLMs on a cleaned 1,195-image six-way benchmark using numeric, visual-cue, and Diwani-first prompts.
- In this closed-set setting, the best numeric-prompt accuracy is 60.9% for closed/API models and 47.2% for open-weight models.
- Prompting helps unevenly: visual cues move accuracy by more than 10 points for some models but hurt others, and foregrounding Diwani often relocates the default rather than removing it.
- Arabic calligraphy recognition is therefore a compact, culturally grounded stress test for VLM robustness and for measuring which visual traditions become legible to automated systems.
Sources (1)
- [1]When Models Misread Ink: Auditing VLMs for Arabic Calligraphy RecognitionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 02:58 PM
Arabic calligraphy is a high-stakes cultural visual domain for Muslim communities, yet VLM evaluations rarely test its fine-grained script traditions.
We audit 12 VLMs on a cleaned 1,195-image six-way benchmark using numeric, visual-cue, and Diwani-first prompts.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026Beyond Generation: Using Multimodal LLMs as Detectors of LLM-Generated Deepfake Facial Images
- Oct 15, 2026ContextLift: Breaking the Spurious Correlation Behind Conditional Emergent Misalignment
- Oct 15, 2026MYOPIA: Understanding the Limits of Tree-Based Reasoning in Vision-Language Models
- Oct 14, 2026What Happens to the Goal? Distinct Failures of Policy Alignment in Goal Misgeneralization
- Oct 14, 2026Vision-Language Models Integrate Conflicting Evidence Like Biased Bayesian Observers