Probe-and-Resume: When and How to Evaluate Agent Execution

Key points

• We study this approach in terminal-based computer-use environments, which expose only a partial view of a large underlying state, and find that the Monte Carlo tree search (MCTS) configuration we evaluate does not significantly improve task success.1
• We identify two sources of poor critic performance: the agent's observations omit environment state needed for evaluation, and intermediate states offer limited evidence of whether a trajectory will succeed.1
• We study when to invoke a critic and what access to give it to evaluate a node.1
• We call the resulting design Probe-and-Resume: when the agent claims to be done, the critic probes a fork of its state, and a rejected agent resumes where it stopped.1

Sources (1)

• [1]Probe-and-Resume: When and How to Evaluate Agent Execution
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 01:42 AM

  Test-time search can improve agent performance by exploring alternative actions and using a critic to select promising branches.

  We study this approach in terminal-based computer-use environments, which expose only a partial view of a large underlying state, and find that the Monte Carlo tree search (MCTS) configuration we evaluate does not significantly improve task success.

Extractive summary: sentences quoted from the sources.