OR-License: Evaluating LLM Agents as Real-World Operations Research Practitioners

Key points

• Existing OR benchmarks typically begin with self-contained descriptions or predefined optimization tasks, underrepresenting the translation of original business materials into professionally validated solutions.1
• We introduce OR-License, an end-to-end benchmark of 31 instances across 10 industrial scenarios from paid OR engagements totaling USD 3.68 million.1
• To study information acquisition, we introduce ORSim, a controlled client simulator that discloses business facts without supplying OR expertise.1
• Across 11 frontier LLMs using coding-agent harnesses, several exceed the expert reference in conditional objective quality, but none matches its DAQ; the strongest primary Claude Code configuration achieves 88.39% strict feasibility and a DAQ of 0.87 against the expert reference of 1.1

Sources (2)

• [1]OR-License: Evaluating LLM Agents as Real-World Operations Research Practitioners
  Hugging Face Daily Papers · Oct 16, 12:00 AM

  Can large language model (LLM) agents deliver expert-level operations research (OR) solutions directly from original client materials, without prior expert reformulation?

  Existing OR benchmarks typically begin with self-contained descriptions or predefined optimization tasks, underrepresenting the translation of original business materials into professionally validated solutions.

• [2]OR-License: Evaluating LLM Agents as Real-World Operations Research Practitioners
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 06:13 PM · same content

Extractive summary: sentences quoted from the sources.