Back on Track: Deployment-time Policy Improvement via World Model Planning

Key points

• We instead propose an offline-trained monitoring-and-intervention framework that improves a policy during deployment using only black-box query access.1
• From a limited set of labeled on-policy rollouts, we learn a policy-conditioned liveness value that measures the nominal policy's progress toward task completion; we additionally train a compact latent world model using these rollouts together with play data.1
• When it detects a loss of progress, a short-horizon planner searches the world model for corrective actions using the same value as its planning objective, after which control returns to the nominal policy.1
• We evaluate our method on the LIBERO-Goal task suite and on a Franka Panda manipulator, using the vision-language-action model $π{0.5}$ and a diffusion policy as nominal policies, and demonstrate significant improvements in task success rates, while supporting real-time control.1

Sources (1)

• [1]Back on Track: Deployment-time Policy Improvement via World Model Planning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 04:48 AM

  State-of-the-art visuomotor policies are increasingly capable of executing complex tasks, yet can fail during deployment in ways they cannot predict or recover from on their own.

  We instead propose an offline-trained monitoring-and-intervention framework that improves a policy during deployment using only black-box query access.

Extractive summary: sentences quoted from the sources.