How Representational Priors Shape Generalization in World Action Models

Key points

• World action models (WAMs) are shown to substantially diverge under generalization performance based on their generative backbones.1
• We investigate this gap through the representational geometry that WAMs inherit from their backbones and how policy training reorganizes it.1
• Our central hypothesis is that a useful representational prior would structure geometry that separates nuisance information from the in-distribution manifold in advance of training as a policy.1
• Together, these findings provide a geometric perspective on how the representational prior of the world model shapes the generalization performance of WAMs.1

Sources (1)

• [1]How Representational Priors Shape Generalization in World Action Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 03:59 AM

  Across various image and video backbones, we show that a higher orthogonality from the representational prior is associated with stronger robustness to visual variations after WAM training, enabling policy generalization measurable prospectively for world models.

  World action models (WAMs) are shown to substantially diverge under generalization performance based on their generative backbones.

Extractive summary: sentences quoted from the sources.