World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models

Key points

• Video world models must generate physically plausible futures, not just realistic frames.1
• Existing approaches typically use semantic, physical, or predictive information to guide the visual generation stream, leaving a fundamental question: where inside the generator should predictions of scene evolution live?1
• Standard video tokens jointly encode appearance and dynamics, but provide no distinct state for learning and reading predictive dynamics.1
• In a controlled comparison of predictive-state interfaces on Physics-IQ Verified, Physical Registry improves vanilla fine-tuning by 9.9/8.7 points on I2V/V2V, the largest gain among the interfaces we compare.1

Sources (1)

• [1]World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 08:18 AM

  We answer this question with the Physical Registry: temporally grounded registry slots inside a video DiT that are initialized from an observation-conditioned forecast, co-evolve with video tokens, and receive future-representation supervision, making predictive dynamics an explicit, readable state of the generator.

  Video world models must generate physically plausible futures, not just realistic frames.

Extractive summary: sentences quoted from the sources.