Agent World Model in the Wild: Computer-Use Agent Environment Synthesis

Key points

• Agent learning is shifting from learning from human data toward learning through interaction with the world.1
• However, scaling such learning directly in real-world computer systems is challenging and can introduce substantial security risks.1
• Whereas existing approaches typically specialize in a single agent interaction interface, we construct executable environments that jointly support graphical user interfaces, model context protocol services, and command-line interfaces, allowing agents to flexibly combine visual interaction, tool use, and code execution.1
• Beyond environments, we address three recurring challenges in task synthesis: ensuring solvability, maintaining sufficient difficulty, and producing reliable verifiers, by introducing execution-based quality gates that combine heterogeneous agent rollouts, cross-judge consistency checks, and counterfactual probes of judgment robustness.1

Sources (2)

• [1]Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 05:55 PM

  We therefore propose Agent World Model in the Wild (WildAWM), a scalable pipeline for synthesizing controllable environments and tasks for training general computer-use agents (CUAs).

  Agent learning is shifting from learning from human data toward learning through interaction with the world.

• [2]Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
  Hugging Face Daily Papers · Oct 19, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.