De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control

Key points

• We ask what a single-step predicted future must preserve to support accurate single-step action prediction.1
• Video-space supervision only partially closes this gap, and a local sensitivity analysis shows that vision prediction errors have highly unequal consequences for action accuracy.1
• These findings motivate DirectWAM, which converts pretrained video and action experts into single-step predictors without stochastic sampling through action-sufficient distillation, while retaining the explicit future-to-action cascade.1
• After learning a one-step action model, we train the future predictor by decoding actions from its own predicted futures and backpropagating demonstration action error through the expert, with an optional future-latent reconstruction loss.1

Sources (1)

• [1]De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 12:30 PM

  Cascade world-action models (WAMs) use predicted visual futures to condition robot actions, but inherit costly iterative inference from diffusion-pretrained experts.

  We ask what a single-step predicted future must preserve to support accurate single-step action prediction.

Extractive summary: sentences quoted from the sources.