Reading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loops

Key points

• This makes them fast and cheap, but when they choose wrongly there is nothing to inspect: did the model misread what the agent has seen, or read it correctly and still act wrongly?1
• On a frozen benchmark of agent decision states from simulated environments, customer-service policies, tool use, web navigation and coding, we pair every next-action question with factual probes about the same state, each answered independently.1
• Across seventeen fast systems and a reasoning model, the answer is consistent: most wrong actions follow correct reading, and the more accurate a system, the more of its errors take this form.1
• Where the probed facts provably determine the action, models that read every fact still fail to combine them, and their own answers, composed by code, repair the failure.1

Sources (2)

• [1]Reading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loops
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 03:43 PM

  System One models choose an agent's next action by scoring a supplied set of options in a single pass.

  This makes them fast and cheap, but when they choose wrongly there is nothing to inspect: did the model misread what the agent has seen, or read it correctly and still act wrongly?

• [2]Reading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loops
  Hugging Face Daily Papers · Oct 13, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.