Structured Latent Dynamics for Long-Horizon Embodied World Models

Key points

• Although recent generative models can synthesize visually plausible action-conditioned futures, their latent dynamics are typically learned through unconstrained transition modules, which may gradually deviate from coherent physical evolution as predictions are recursively rolled forward.1
• We investigate whether explicitly structuring latent state evolution can improve the stability and utility of such imagined trajectories.1
• To this end, we introduce HWM, a structured dynamics framework that represents backbone latents in a learned phase space consisting of generalized coordinates and momenta.1
• We further evaluate the approach across three distinct embodied world-model settings, covering single-arm multi-view prediction with Ctrl-World, dual-arm single-view prediction with EVAC, and task-oriented single-arm prediction with Cosmos-Predict2.5-2B.1

Sources (1)

• [1]Structured Latent Dynamics for Long-Horizon Embodied World Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 07:54 AM

  Reliable long-horizon prediction remains a central challenge for embodied world models.

  Although recent generative models can synthesize visually plausible action-conditioned futures, their latent dynamics are typically learned through unconstrained transition modules, which may gradually deviate from coherent physical evolution as predictions are recursively rolled forward.

Extractive summary: sentences quoted from the sources.