ProteinGeoBench: Frontier Structural-Biology Agents Are Fluent at Lookup but Lost in Geometry
Frontier agents retrieve structural metadata well and compute geometry from coordinates less reliably.
ProofPaper ↗
Key points
- We ask how much of a benchmark score reflects that gap and how much reflects the evaluation configuration around the model.
- We study ProteinGeoBench, a suite of 131 protein-structure tasks, using 4,249 rollouts from four frontier models and 4,486 further trials across 26 models.
- The fourth is slightly better on numeric.
- The experiments use different task subsets, so they cannot rank model, harness, access and task composition against one another.
Sources (1)
- [1]ProteinGeoBench: Frontier Structural-Biology Agents Are Fluent at Lookup but Lost in GeometryarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 05:55 PM
Frontier agents retrieve structural metadata well and compute geometry from coordinates less reliably.
We ask how much of a benchmark score reflects that gap and how much reflects the evaluation configuration around the model.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Nvidia launches new open models to run agents in the enterprise and on PCs
- Oct 19, 2026MiniMax M3.1 Flash shows China's AI price race has no floor in sight
- Oct 19, 2026Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
- Oct 19, 2026Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
- Oct 19, 2026AnyGoal: On World Model Generalization Beyond Expert Rollouts
- Oct 18, 2026CoEvo-RSI: When Harnesses Evolve, Which Models Should Follow?