Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?

Key points

• Frontier coding agents are increasingly framed as end-to-end automated researchers, and ML for science offers a demanding test of that promise: improving a scientific model often requires domain insight beyond routine ML engineering.1
• Rather than ranking agents on a leaderboard, Agent4Sci asks what kinds of progress agents can make on these tasks under in-the-wild conditions, and where they fall short.1
• Trajectory analysis surfaces three recurring limitations in this setting: novelty miscalibration, resource under-utilization, and rapid collapse into local engineering sweeps.1
• Agent4Sci gives the community a way to track and push frontier agents from ML engineering toward domain insight on real-world ML-for-science problems.1

Sources (2)

• [1]Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?
  Hugging Face Daily Papers · Oct 20, 12:00 AM

  We introduce Agent4Sci, a long-horizon testbed for measuring how far today's agents can improve the published best models on scientific ML benchmarks, spanning 20 benchmarks across 11 domains.

  Frontier coding agents are increasingly framed as end-to-end automated researchers, and ML for science offers a demanding test of that promise: improving a scientific model often requires domain insight beyond routine ML engineering.

• [2]Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 06:03 PM · same content

Extractive summary: sentences quoted from the sources.