Back Up Before You Regret: Measuring Backup Awareness and Automating Backup in Coding Agents

Key points

• Coding agents now complete tasks autonomously, modifying real-world state such as artifacts, software environments, and configurations along the way.1
• Yet they show limited backup awareness: recognizing that an action will modify state that may later be needed, and preserving it beforehand.1
• Across six frontier models, agents complete 86.1% of tasks on average yet back up before these moments in only 15.2%, showing that task completion ability does not imply backup awareness.1
• Because it is hard to predict which changes will be regretted, we propose BackUpper, a harness plug-in backup layer that checks every tool call and preserves affected state before modification.1

Sources (1)

• [1]Back Up Before You Regret: Measuring Backup Awareness and Automating Backup in Coding Agents
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 09:13 PM

  To evaluate backup awareness, we introduce RegretBench, a benchmark of 205 executable tasks grounded in regret moments reported on platforms such as X.com, GitHub, and Stack Overflow, where users explicitly regretted not preserving state they later needed.

  Coding agents now complete tasks autonomously, modifying real-world state such as artifacts, software environments, and configurations along the way.

Extractive summary: sentences quoted from the sources.