Answering in the User's Variety: Calibrated Accommodation Pairs for Post-Training

Key points

• People adapt how they talk to the person in front of them.1
• However, chat models tend to respond in Standard English regardless of the dialect used by the user.1
• We trace how each post-training stage, from supervised fine-tuning (SFT) to preference optimization (DPO) and reinforcement learning, changes a model's tendency to match the user, and read what each update prefers from two content-matched replies, one matching the user and one not.1
• Across 10 open model lineages, post-training standardizes replies to every informal cue, yet after adjusting for how likely each reply already was, the DPO update still favours matching British and Australian spelling over AAE and Singlish on 10 lineages, and models still produce AAE features when asked.1

Sources (2)

• [1]Answering in the User's Variety: Calibrated Accommodation Pairs for Post-Training
  Hugging Face Daily Papers · Oct 20, 12:00 AM

  We then introduce calibrated accommodation pairs (CAP), preference data that reward matching users' dialect density while preserving Standard English for Standard-English users.

  People adapt how they talk to the person in front of them.

• [2]Answering in the User's Variety: Calibrated Accommodation Pairs for Post-Training
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 04:26 PM · same content

Extractive summary: sentences quoted from the sources.