LIBERO-Pro2.0: A Benchmark for Evaluating the Robustness of Embodied Foundation Models

Key points

• LIBERO has emerged as a widely adopted benchmark for evaluating robotic manipulation policies.1
• However, strong performance on LIBERO does not necessarily indicate robust embodied capabilities, as models may exploit benchmark-specific regularities rather than generalize to diverse operating conditions.1
• LIBERO-Pro2.0 encompasses 22 static perturbations and 20 runtime interventions across six dimensions: Condition, Environment, Observation, Execution, Robot State Configuration, and Language, yielding 1,648 validated shifted task instances.1
• We evaluate 17 representative robot policies spanning vision-language-action models, world-action models, and robustness-oriented policies.1

Sources (1)

• [1]LIBERO-Pro2.0: A Benchmark for Evaluating the Robustness of Embodied Foundation Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 09:48 AM

  In this work, we introduce LIBERO-Pro2.0, a comprehensive benchmark for systematically evaluating the robustness of embodied foundation models under distribution shifts.

  LIBERO has emerged as a widely adopted benchmark for evaluating robotic manipulation policies.

Extractive summary: sentences quoted from the sources.