Coopshift: Post-Training Amplifies The Memory Curse In LLM Cooperation

Key points

• Within each of seven released model lineages, we compare pretrained (Base), supervised fine-tuned (SFT), and subsequent preference- or reward-optimized (Aligned) checkpoints across seven social dilemmas.1
• The memory curse refers to lower cooperation when agents can see more past rounds.1
• With long history, both post-trained stages also cooperate less than Base on average in each binary-action game.1
• The difference in cooperation rates between Aligned and SFT also changes from one-shot decisions to repeated play.1

Sources (1)

• [1]Coopshift: Post-Training Amplifies The Memory Curse In LLM Cooperation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 06:11 PM

  How does post-training change cooperation during repeated interaction?

  Within each of seven released model lineages, we compare pretrained (Base), supervised fine-tuned (SFT), and subsequent preference- or reward-optimized (Aligned) checkpoints across seven social dilemmas.

Extractive summary: sentences quoted from the sources.