VPFlow: Value-Predictive Planning with Imperfect World Models for Offline Goal Reaching

Key points

• Model-based methods allow for planning by scaling test-time compute to solve complex tasks.1
• In this work, we show that value prediction is more useful for planning, whereas accurate state prediction is often unnecessary.1
• We use this return as the sole supervision for latent world models: imagined latents are trained to predict goal-reaching value rather than match future states, implicitly preventing representation collapse.1
• We then present VPFlow, a value-predictive planner that pairs the world models with a one-step flow policy and plans in its noise space via evolutionary search.1

Sources (1)

• [1]VPFlow: Value-Predictive Planning with Imperfect World Models for Offline Goal Reaching
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 08:55 PM

  Offline goal-conditioned reinforcement learning enables skill acquisition by reaching any goal from any state without dense rewards.

  Model-based methods allow for planning by scaling test-time compute to solve complex tasks.

Extractive summary: sentences quoted from the sources.