FrontierCS 2: A Benchmark for Computer Science R&D
We introduce FrontierCS 2, a benchmark of 48 open-ended tasks.
ProofPaper ↗
Key points
- Can LLM agents make contributions to frontier computer science R&D?
- We take a paper and the code its authors released and delete the paper's contribution.
- Earlier open-ended benchmarks give each agent a problem statement and grade one program after the run.
- FrontierCS 2 places the agent inside a research environment.
Sources (1)
- [1]FrontierCS 2: A Benchmark for Computer Science R&DarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 04:04 PM
We introduce FrontierCS 2, a benchmark of 48 open-ended tasks.
Can LLM agents make contributions to frontier computer science R&D?
Extractive summary: sentences quoted from the sources.
Before this
- Oct 21, 2026Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets
- Oct 20, 2026Voxtral 3
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 20, 2026Cache Engineering for Agents
- Oct 20, 2026Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling
- Oct 19, 2026Evaluating Agents on Highly Subjective Tasks