Auditing the Product, Not the Recursion: The Evaluation Evidence Behind Recursive Self-Improvement Claims

Key points

• Evaluating this promise requires separating product claims about the final system from process claims about the improvement process itself.1
• The codebook links claims to evaluation designs.1
• Independent human coding, third-reader adjudication, and a separate blind coding of improver comparisons assess coding reliability.1
• Three propositions explain how selection effects, shared evaluator errors, and changes in system state limit what product evaluations can establish.1

Sources (1)

• [1]Auditing the Product, Not the Recursion: The Evaluation Evidence Behind Recursive Self-Improvement Claims
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 04:23 AM

  Recursive self-improvement (RSI) promises AI systems, such as self-evolving LLM agents, that become better at producing further improvements.

  Evaluating this promise requires separating product claims about the final system from process claims about the improvement process itself.

Extractive summary: sentences quoted from the sources.