Auto-MD-bench: Evaluating Autoresearch Agents for Real-World Medical Tasks
In this study, we introduce Auto-MD-bench, a benchmark of 15 real-world medical tasks from 10 clinical datasets, spanning survival analysis, classification and sequential prediction, with external validation across cohorts, hospitals and time periods.
ProofPaper ↗
Key points
- Autonomous agents are increasingly capable of developing machine learning (ML) models end to end.
- For example, medical ML models should also provide well-calibrated risk estimates and remain robust across patient populations.
- This raises a key question: are these objectives sufficient to guide agents toward models that meet the broader requirements of real-world medical use?
- We evaluate four agents: AIDE, AI Scientist v2, Codex, and Autoresearch.
Sources (1)
- [1]Auto-MD-bench: Evaluating Autoresearch Agents for Real-World Medical TasksarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 07:19 AM
In this study, we introduce Auto-MD-bench, a benchmark of 15 real-world medical tasks from 10 clinical datasets, spanning survival analysis, classification and sequential prediction, with external validation across cohorts, hospitals and time periods.
Autonomous agents are increasingly capable of developing machine learning (ML) models end to end.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 17, 2026Do Conversational Agents Need Explicit Memory? A Controlled Comparison with Raw-History Retrieval
- Oct 17, 2026MELD: Recursive Self-Improvement via Hierarchical Coordination for Multi-Agent Scientific Discovery
- Oct 17, 2026Can Agents Keep Secrets They Were Never Told? Benchmarking Latent Privacy Risks Across Sessions with MemJoin
- Oct 17, 2026Design Anything, Design at Scale
- Oct 16, 2026SteerSearch: Interactive Search Reasoning with Mid-Response Steering
- Oct 16, 2026LoopHIL: Looping Hardware into LLM-assisted Agents for Wireless Embedded Systems