Frontier LLMs Enter the World of Robotics: A Cross-Model Evaluation on RoboColiseum

Key points

• We conduct a multidimensional comparative evaluation on RoboColiseum across four tracks: Instruction Following, Spatial Reasoning, General Manipulation, and Robustness.1
• We implement Agent2Act, a control harness that integrates GPT-6 Astra, GPT-6.1 Sol, Claude Opus 5.5, and Gemini 4 Argon into the evaluation environment through shared observation, action execution, and feedback interfaces.1
• These results reveal complementary strengths across models and a persistent gap between task understanding and reliable physical execution.1
• Additionally, case studies with Astra examine whole-body coordination, manipulation across robot embodiments, precision manipulation, dexterous-hand manipulation, and real-robot tasks, providing further insight into the model's capabilities and limitations across embodied tasks.1

Sources (1)

• [1]Frontier LLMs Enter the World of Robotics: A Cross-Model Evaluation on RoboColiseum
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 08:46 AM

  Recent studies have explored the potential of multimodal large language models for robot control, yet the models' embodied capabilities remain insufficiently characterized across models, tasks, and execution settings.

  We conduct a multidimensional comparative evaluation on RoboColiseum across four tracks: Instruction Following, Spatial Reasoning, General Manipulation, and Robustness.

Extractive summary: sentences quoted from the sources.