Cooperation Without Shared Rewards via Reciprocal Q-Augmentation

Key points

• General-sum games are more challenging than fully cooperative or adversarial settings because agents may deceive or exploit one another.1
• Achieving high social welfare requires sustained cooperation, while a single defection can drive agents toward a welfare suboptimal Nash equilibrium.1
• To address this issue, we introduce Reciprocal Q-Augmentation (RQA), a method that uses a latent-variable opponent model to augment the Q-function with estimates of how one's actions affects other agents.1
• Our experiments show that the algorithms implementing RQA outperforms fully cooperative methods with shared rewards and enables cooperation with algorithms that otherwise fail to reach high social welfare.1

Sources (1)

• [1]Cooperation Without Shared Rewards via Reciprocal Q-Augmentation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 05:54 PM

  We show that, in social dilemmas, agents that all follow the cooperative policy of RQA learn to cooperate faster and reach higher levels of cooperation than fully cooperative algorithms.

  General-sum games are more challenging than fully cooperative or adversarial settings because agents may deceive or exploit one another.

Extractive summary: sentences quoted from the sources.