Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience

Key points

• World models enable embodied agents to anticipate action consequences, yet jointly modeling locomotion and manipulation across diverse embodiments remains challenging due to heterogeneous action spaces and limited multimodal supervision.1
• A unified kinematic representation encodes locomotion through camera poses and manipulation through projected skeletons, with robot skeletons combining embodiment-specific forward kinematics with extensive manual keypoint annotation and correction.1
• We also introduce Xp-Teleop, a long-horizon loco-manipulation benchmark based on high-quality dexterous humanoid teleoperation data.1
• Evaluations demonstrate state-of-the-art performance across five cross-embodiment benchmarks (DreamDojo, $Ψ0$, $ω$-HOME, Xp-Teleop, and AgiBot) and on held-out human egocentric video, from zero-shot inference to few-shot and full-data post-training.1

Sources (2)

• [1]Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 05:59 PM

  We present Ropedia W0, the first egocentric loco-manipulation world foundation model spanning humans and robots with diverse embodiments.

  World models enable embodied agents to anticipate action consequences, yet jointly modeling locomotion and manipulation across diverse embodiments remains challenging due to heterogeneous action spaces and limited multimodal supervision.

• [2]Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
  Hugging Face Daily Papers · Oct 19, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.