From Seeing to Reporting: A Theory of Reliable Visual Set Generation in Vision-Language Models
Vision-language models (VLMs) are asked to report what they see: to count objects, box them, parse a screen or pick a click target (GUI grounding).
ProofPaper ↗
Key points
- We argue that reporting is part of the visual task, not merely its formatting.
- In a dense scene with 173 shapes, GPT-6.1 lists 173 records that match every shape, yet the counts it states in the same response sum to 188.
- From Seeing to Reporting (FSR) is a theory of visual reporting and a set of inference-time procedures that improve the reports of frozen VLMs. It models reporting as a chain from responses to decoded records, constructed sets and task answers, linked by task-compatible matching.
- On 50 dense synthetic scenes, a one-call reporting-and-decoding protocol cuts per-type count error by 85.2% for GPT-6.1 and 98.8% for Claude Sonnet 5.5.
Sources (1)
- [1]From Seeing to Reporting: A Theory of Reliable Visual Set Generation in Vision-Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 07:43 PM
Vision-language models (VLMs) are asked to report what they see: to count objects, box them, parse a screen or pick a click target (GUI grounding).
We argue that reporting is part of the visual task, not merely its formatting.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 18, 2026Think Visually Across Space and Time: Grounded Reasoning for Video Segmentation
- Oct 18, 2026Painting Change Map as guidance for generating painting processes
- Oct 17, 2026TopoCompile: Molecular Generation with Exact Global Count Constraints
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026Momentum: Persistent Dynamics for Video World Models
- Oct 16, 2026ReLaDrift: Recurrent Language Generation via Drifting for Test-Time Scaling