RoboFQE: Rollout-Free Evaluation of Visual Robot Policies from Offline Trajectories

Key points

• Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models.1
• In this paper, we study off-policy evaluation as an alternative to rollout-based robot-policy evaluation methods.1
• Controlled experiments across simulated and real-world manipulation tasks show that RoboFQE produces accurate discounted-return estimates and preserves policy rankings even on long-horizon manipulation tasks where video world-model evaluators are degraded by autoregressive prediction errors.1
• Ablation experiments show that pretrained vision encoders substantially outperform from-scratch alternatives, particularly on long-horizon tasks, and that learned terminal success annotation enables evaluation without ground-truth reward labels with only slight reduction of accuracy.1

Sources (1)

• [1]RoboFQE: Rollout-Free Evaluation of Visual Robot Policies from Offline Trajectories
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 04:48 PM

  Specifically, we introduce RoboFQE, an evaluator that estimates the discounted expected return of a target policy from offline robot trajectories without estimating a behavior-policy density, combining fitted Q-evaluation with a pretrained vision foundation model and learned terminal success annotation to bridge the gap between standard OPE benchmarks and continuous-control visual manipulation.

  Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models.

Extractive summary: sentences quoted from the sources.