MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering

Key points

• Large language models (LLMs) are expanding from assisting scientific research and engineering to autonomously developing and improving AI systems, advancing AI for AI (AI4AI) as a pathway toward recursive self-improvement (RSI).1
• Workflow-driven machine learning engineering (MLE) systems provide a practical path toward this pursuit through complex orchestration that manages long-horizon experimentation, while recent MLE systems adopt model-directed interaction, allowing models to autonomously organize experiments and adapt optimization strategies based on execution feedback.1
• We design the Kaggle-MLE Factory to construct executable tasks and an interaction harness for model-directed experimentation.1
• Hierarchical agentic SFT transfers general tool-use and coding capabilities to MLE through outcome-guided supervision.1

Sources (2)

• [1]MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 05:56 PM

  We propose a post-training method for autonomous long-horizon MLE optimization and apply it to train MLExpert, a 35B Mixture-of-Experts agentic model.

  Large language models (LLMs) are expanding from assisting scientific research and engineering to autonomously developing and improving AI systems, advancing AI for AI (AI4AI) as a pathway toward recursive self-improvement (RSI).

• [2]MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
  Hugging Face Daily Papers · Oct 15, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.