LADDER: A Long-Horizon Robot Manipulation Benchmark

Key points

• General-purpose robot manipulation policies learn short-horizon skills effectively, but degrade rapidly on long-horizon tasks.1
• Existing benchmarks cannot isolate failures because task horizon is either too short or entangled with scene and object variations.1
• We evaluate two orchestration paradigms: text subgoals for learned low-level policies (VoLo) and tool selection orchestration (CaP-X, GPT-Policy, PSP-Agent, TiPToP).1
• Tool selection orchestration with GPT-Policy on GPT-6 Astra significantly outperforms learned-policy methods, but incurs significant time and cost budget and succeeds only 62%.1

Sources (1)

• [1]LADDER: A Long-Horizon Robot Manipulation Benchmark
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 09:42 AM

  We propose LADDER, a long-horizon robot manipulation benchmark scaling from 1 to 10 subgoals within the same environment over three suites (SimplerEnv-Long, RobotArena-$\infty$-Long, and RoboLab-Long), 145 tasks, eight categories, two embodiments, and 2,024 simulated episodes per system.

  General-purpose robot manipulation policies learn short-horizon skills effectively, but degrade rapidly on long-horizon tasks.

Extractive summary: sentences quoted from the sources.