When Coding Agents Outpace Review: Evaluating PRD-Driven Development in Mature Enterprise Repositories
Coding agents increasingly undertake development tasks in mature enterprise repositories, yet existing benchmarks offer limited insight into whether they fulfill complete Product Requirement Documents (PRDs) and where delivery breaks down.
ProofPaper ↗
Key points
- Issue resolution and development from scratch leave incremental PRD fulfillment underexplored, while isolated tests provide incomplete evidence of business and integration requirements.
- We introduce PACE, a benchmark of 43 enterprise tasks with 750 expert-audited Code Points.
- It combines requirement reconstruction, criterion construction, and evidence-based evaluation to measure PRD fulfillment and connect delivery gaps to agent behavior.
- Incomplete implementation and missing integration are the principal bottlenecks: producing individual components does not ensure a complete delivery.
Sources (1)
- [1]When Coding Agents Outpace Review: Evaluating PRD-Driven Development in Mature Enterprise RepositoriesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 01:39 PM
Coding agents increasingly undertake development tasks in mature enterprise repositories, yet existing benchmarks offer limited insight into whether they fulfill complete Product Requirement Documents (PRDs) and where delivery breaks down.
Issue resolution and development from scratch leave incremental PRD fulfillment underexplored, while isolated tests provide incomplete evidence of business and integration requirements.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026VibeTraining-Bench: Data-Trainer-Harness Co-Design for Iterative System Improvement
- Oct 19, 2026MiniMax M3.1 Flash shows China's AI price race has no floor in sight
- Oct 19, 2026Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
- Oct 18, 2026Google Open-Sources Harness2, a Recursive Agent Harnessing Framework
- Oct 17, 2026Design Anything, Design at Scale
- Oct 16, 2026AM-Bench: A Unified Taxonomy and Evaluation Suite for Agentic Misalignment