EgoLAP: Learning from Egocentric Human Data through Language-Action Reasoning

Key points

• Egocentric human data offer a path to scaling robot learning beyond costly robot demonstrations, yet the embodiment gap makes raw human trajectories a poor supervisory target for control.1
• Our key insight is that, although low-level actions are embodiment-specific, their underlying motion intent can capture task-relevant structure that transfers across humans and robots.1
• EgoLAP expresses motion intent as structured, temporally abstracted language actions and pairs them with motion-level reasoning grounded in scene geometry, physics, and object affordances.1
• Across extensive real-world and simulated experiments, EgoLAP transfers human experience to robot control more effectively than alternative action representations and reaches 80.1% mean real-world task progress, a 2.3x performance gain over alternative action representations.1

Sources (1)

• [1]EgoLAP: Learning from Egocentric Human Data through Language-Action Reasoning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 05:30 PM

  We introduce EgoLAP, a VLA pre-training framework that jointly learns from human and robot trajectories through a shared language-based action chain-of-thought.

  Egocentric human data offer a path to scaling robot learning beyond costly robot demonstrations, yet the embodiment gap makes raw human trajectories a poor supervisory target for control.

Extractive summary: sentences quoted from the sources.