Hard Instances on Demand
We develop a widely applicable workflow built on the Nevergrad optimization platform and demonstrate it on four NP-hard problems: satisfiability, traveling salesperson, Hamiltonian cycle, and non-permutation flow-shop scheduling.
ProofPaper ↗
Key points
- The conclusions of an experimental study of algorithms are only as good as the instances they are run on.
- Benchmark libraries for well-studied problems took decades to curate, while evaluations of new problems often fall back on random instances, which are known to be misleading: they may say more about the random generator than about the algorithmic question at hand.
- We treat instance generation as an optimization problem: parameterize the instance space, take the solver's running time as the objective, and let a black-box optimizer search for hard instances.
- Our evolved instances beat the hardest instance of a budget-matched random draw in all four case studies, by an order of magnitude or more in three.
Sources (1)
- [1]Hard Instances on DemandarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 08:47 PM
We develop a widely applicable workflow built on the Nevergrad optimization platform and demonstrate it on four NP-hard problems: satisfiability, traveling salesperson, Hamiltonian cycle, and non-permutation flow-shop scheduling.
The conclusions of an experimental study of algorithms are only as good as the instances they are run on.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint
- Oct 15, 2026ETH: Data-Balanced Heterogeneous Pipeline Parallelism for Training Large Models
- Oct 15, 2026Noise-Space Reward Alignment for Few-Step Generative Models
- Oct 15, 2026Welcome FLUX.3 Image Klein in Diffusers
- Oct 15, 2026Has GPT-6-Astra Achieved Embodied Intelligence?
- Oct 14, 2026Crossing the Threshold: How OpenAI and Anthropic Approach Autonomy and Safety in Financial Agents