Identity, Incentives, and Cooperative Behavior for AI: Reverse-Engineering Multiagent Cooperation

Key points

• The behavior of AI systems increasingly depends on the extent to which agents cooperate or coordinate with one another, in ways that may be desirable or not depending on the situation.1
• Conventional explanations for cooperative behavior distinguish between purely other-regarding preferences ("altruism") and instrumental reasons for self-interested agents to cooperate.1
• The way that agents should enact these mechanisms for cooperation (or not) is complicated by the fact that there is not a clear or unambiguous locus of identity or selfhood for AI agents.1
• We argue that many different mechanisms can often produce similar observed behaviors and introduce a set of experimental designs along with conditions under which they can successfully disentangle competing explanations for multiagent behavior.1

Sources (1)

• [1]Identity, Incentives, and Cooperative Behavior for AI: Reverse-Engineering Multiagent Cooperation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 08:52 PM

  We develop a framework for reverse-engineering why a given set of agents engage in a particular set of cooperation-related behaviors.

  The behavior of AI systems increasingly depends on the extent to which agents cooperate or coordinate with one another, in ways that may be desirable or not depending on the situation.

Extractive summary: sentences quoted from the sources.