ToM2Vec: Continuous Theory-of-Mind Tokens for Multi-Agent Coordination

Key points

• Forming an accurate theory-of-mind (ToM) is key to successful multi-agent collaboration and requires reasoning under uncertainty to capture multiple possible partner states.1
• Across several environments covering collaborative, competitive, and mixed-motive settings (Overcooked-AI, multiparty negotiation, Decrypto, Hanabi, and Collab Escape/Capture), we evaluate ToM2Vec's ability to support multi-agent interactions and collaboration, finding consistent improvements over the base model and a variety of competitive trained and prompted baselines.1
• We show that latent reasoning is critical, with performance dropping when reasoning is expressed in text.1
• In our analysis, we provide evidence that the latent tokens encode a ToM, rather than task-specific information: ToM2Vec's tokens improve the agent's ability to directly answer questions about partner beliefs, desires, and next actions.1

Sources (2)

• [1]ToM2Vec: Continuous Theory-of-Mind Tokens for Multi-Agent Coordination
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 04:05 PM

  To address this challenge, we introduce ToM2Vec, which learns to form abstract theories-of-mind in a rich latent space via a fixed budget of continuous reasoning tokens.

  Forming an accurate theory-of-mind (ToM) is key to successful multi-agent collaboration and requires reasoning under uncertainty to capture multiple possible partner states.

• [2]ToM2Vec: Continuous Theory-of-Mind Tokens for Multi-Agent Coordination
  Hugging Face Daily Papers · Oct 21, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.