Beyond Perfect Teammates: Multi-Agent Collaboration with Unreliable Colleagues

Key points

• Yet existing systems and evaluations largely rest on an implicit perfect-colleague assumption: collaborators provide correct and complete information and respond promptly to requests.1
• In realistic teamwork, however, collaborators may return information that does not answer the intended question, provide insufficient detail to support a decision, or fail to respond to a request.1
• To separate task-solving competence from the ability to recover an unreliable evidence-gathering process, we introduce the Unreliable Colleagues Benchmark (UnCoBench), which organizes behavioral failures through Misaligned, Incomplete, and Missed settings.1
• We also introduce a verifiable and scalable automated generation pipeline that pairs LLM-based scenario generation with executable checks of answer uniqueness and role necessity.1

Sources (2)

• [1]Beyond Perfect Teammates: Multi-Agent Collaboration with Unreliable Colleagues
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 03:31 AM

  LLM-based multi-agent collaboration is increasingly used to solve complex tasks by dividing work among specialized agents and integrating complementary expertise.

  Yet existing systems and evaluations largely rest on an implicit perfect-colleague assumption: collaborators provide correct and complete information and respond promptly to requests.

• [2]Beyond Perfect Teammates: Multi-Agent Collaboration with Unreliable Colleagues
  Hugging Face Daily Papers · Oct 12, 03:31 AM · same content

Extractive summary: sentences quoted from the sources.