Not All Graphs Are Maps: A Task-Agnostic Exploration Framework for Mobile GUI Navigation

Key points

• Modern mobile GUI agents can perceive visible screens and execute local actions, but they still lack persistent app-specific navigation knowledge: where functions live, which pages contain them, and which screens are equivalent across repeated visits.1
• When a future task's destination is covered, TAG-Nav retrieves the destination and provides route guidance without task-specific demonstrations.1
• To evaluate route finding without downstream execution confounds, we introduce DestBench, a navigation-only protocol that asks whether an agent reaches the correct pre-action destination screen.1
• Graph-derived memory improves strict navigation success rate of UI-TARS-1.5-7B from 52.4% without graph memory to 58.3% with automated BFS memory, and improves Qwen-VL-Max success rate from 59.5% to 71.4% on the same conditions.1

Sources (1)

• [1]Not All Graphs Are Maps: A Task-Agnostic Exploration Framework for Mobile GUI Navigation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 09:40 PM

  We introduce TAG-Nav, a Task-Agnostic Graph Navigation framework that wraps existing GUI-agent backbones with reusable destination memory.

  Modern mobile GUI agents can perceive visible screens and execute local actions, but they still lack persistent app-specific navigation knowledge: where functions live, which pages contain them, and which screens are equivalent across repeated visits.

Extractive summary: sentences quoted from the sources.