Frechet Forcing: Training Long-Horizon Streaming World Models from Short Clips

Key points

• For action-conditioned world models, the teacher becomes a bottleneck: an external video model is agnostic to the actions, whereas the model's own pretrained network drifts in the same manner as the student.1
• We propose Fréchet Forcing, which fine-tunes a world model together with its deployed multi-step sampler under three constraints.1
• Generative anchoring applies the flow-matching loss after a self-generated history ending on real data.1
• Context fidelity, a Fréchet distance (FD) on latent statistics of the free rollout centred on each clip's own ground truth, keeps every rollout faithful to its scene.1

Sources (2)

• [1]Frechet Forcing: Training Long-Horizon Streaming World Models from Short Clips
  Hugging Face Daily Papers · Oct 19, 12:00 AM

  Streaming video world models generate the future chunk by chunk, conditioned on their own past and a stream of actions.

  For action-conditioned world models, the teacher becomes a bottleneck: an external video model is agnostic to the actions, whereas the model's own pretrained network drifts in the same manner as the student.

• [2]Frechet Forcing: Training Long-Horizon Streaming World Models from Short Clips
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 02:12 AM · same content

Extractive summary: sentences quoted from the sources.