MASIF: Implicit Failures for Process Observability in Multi-Agent Systems

Key points

• We call these implicit failures (IFs) and study them as signals for process observability beyond final task outcome.1
• We develop taxonomies of error types and subsequent absorption mechanisms, supported by human and cross-model annotation audits.1
• Counterfactual replay further probes how repairing an IF changes the final outcome.1
• To examine the practical value of IFs, we evaluate IF-based best-of-N trace selection against existing process reward models, and introduce a novel RL algorithm, IF-GRPO, that uses IF costs to refine preferences among successful trajectories in GRPO.1

Sources (1)

• [1]MASIF: Implicit Failures for Process Observability in Multi-Agent Systems
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 06:07 AM

  Multi-agent systems (MAS) can complete a task despite local execution errors that are corrected, bypassed, or carried forward without individually determining its outcome.

  We call these implicit failures (IFs) and study them as signals for process observability beyond final task outcome.

Extractive summary: sentences quoted from the sources.