FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business Requests
We introduce FDE-Bench, the first benchmark built from real customer projects to evaluate agents on end-to-end delivery tasks that require interactive requirement clarification and iterative optimization.
ProofPaper ↗
Key points
- Real-world business requests often leave key requirements unstated because frontline practitioners struggle to articulate rules they know from experience.
- Forward-deployed engineers (FDEs) must elicit these requirements through dialogue and iteratively optimize solutions until they meet customer acceptance criteria.
- Existing optimization benchmarks typically start from explicit objectives, while clarification benchmarks often introduce information gaps into existing tasks.
- Agents elicit missing requirements from a simulated customer, and case-specific evaluators assess final artifacts against acceptance criteria using customer-accepted FDE solutions as quality references.
Sources (1)
- [1]FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business RequestsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 03:53 AM
We introduce FDE-Bench, the first benchmark built from real customer projects to evaluate agents on end-to-end delivery tasks that require interactive requirement clarification and iterative optimization.
Real-world business requests often leave key requirements unstated because frontline practitioners struggle to articulate rules they know from experience.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 10, 2026SEPM: Self-Evolving Procedural Memory Grounded in Task Graphs for Multi-Agent Collaboration
- Oct 10, 2026RES-MR: Risk-Aware Reasoning for Explainable and Safe Medication Recommendation
- Oct 9, 2026Proactive Agentic Orchestration
- Oct 9, 2026AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents
- Oct 9, 2026DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design
- Oct 8, 2026Understanding Jev, the new model everyone is talking about