Don't Guess, Just Ask: Evaluating Interactive Requirement Clarification for Repository-Level Software Development
As LLMs become increasingly capable of implementing software, a critical bottleneck emerges: whether they truly understand what users want.
ProofPaper ↗
Key points
- Users often begin with vague intent, leaving important requirements to be clarified through interaction.
- In repository-level development, misunderstandings can propagate and cause costly rework.
- We introduce ReqClarifyBench, an interactive requirement clarification benchmark for repository-level software development.
- We also find that coding-agent harnesses do not further improve requirement document quality compared with their base LLMs. These findings offer insights for building LLM-based end-to-end software development systems.
Sources (1)
- [1]Don't Guess, Just Ask: Evaluating Interactive Requirement Clarification for Repository-Level Software DevelopmentarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 05:23 AM
As LLMs become increasingly capable of implementing software, a critical bottleneck emerges: whether they truly understand what users want.
Users often begin with vague intent, leaving important requirements to be clarified through interaction.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026Qwen/Qwen-Image-2.1-Turbo
- Oct 9, 2026Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices
- Oct 9, 2026The Thinking Leakage in Reinforcement Learning for LLMs
- Oct 9, 2026Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding
- Oct 8, 2026Understanding Jev, the new model everyone is talking about
- Oct 8, 2026Launching an opt-in vulnerability-finding service for open-source software