MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis Search
We propose the Multi-Hypothesis Verification Framework (MHVF) for training-free GUI grounding.
ProofPaper ↗
Key points
- GUI grounding enables web agents to translate natural-language instructions into actionable interface locations.
- It combines multi-hypothesis exploration with local visual verification using a single pretrained vision-language model.
- To explore alternative targets, MHVF constructs spatial hypotheses from multiple full-screen predictions and examines the corresponding regions through zoomed re-localization.
- MHVF improves UI-Venus-Ground-7B by 4.3 and 2.1 percentage points on Mind2Web and VisualWebBench, respectively, and raises Qwen2.5-VL-7B's accuracy on ScreenSpot-Pro from 23.6% to 53.6%.
Sources (1)
- [1]MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis SearcharXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 06:20 AM
We propose the Multi-Hypothesis Verification Framework (MHVF) for training-free GUI grounding.
GUI grounding enables web agents to translate natural-language instructions into actionable interface locations.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026Hybrid Reasoning Agents
- Oct 12, 2026Separating Prompt Harmfulness from Interaction Safety via Partial Generation
- Oct 12, 2026Redesigning source control at Anthropic
- Oct 12, 2026Hashrope: A Persistent Data Structure for Mechanisms of LLM Context Management
- Oct 11, 2026autotrust/JEV-35B-NVFP4
- Oct 11, 2026RSIPO: Recursive Self-Improving Policy Optimization for Agentic Reinforcement Learning via Skill-Policy Co-Evolution