SpecForge: Recursive Self-Improvement for Coding Agents via Behavioral Specifications

Key points

• Coding agents based on frozen language models can improve their performance by refining the external context they rely on, without updating model weights.1
• We formulate a self-improvement loop in which an agent generates code from its current specifications, evaluates it with execution-verified tests, and refines the specifications based on observed failures.1
• The improved specifications then guide subsequent generation, progressively improving the agent's understanding of the codebase.1
• We introduce SpecForge, a multi-frontier tree search framework that addresses this challenge with three mechanisms: (i) Pareto-guided state selection preserves complementary specification states and prevents premature commitment; (ii) dependency-constrained entity selection prioritizes callee-before-caller refinement; and (iii) error-diversified expansion explores multiple refinements using different subsets of test failures.1

Sources (1)

• [1]SpecForge: Recursive Self-Improvement for Coding Agents via Behavioral Specifications
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 03:46 PM

  We study recursive self-improvement through behavioral specifications: natural-language code documentation that describes the behavior of program entities and guides agents in generating correct code.

  Coding agents based on frozen language models can improve their performance by refining the external context they rely on, without updating model weights.

Extractive summary: sentences quoted from the sources.