Steertuning Policies from a Few Human Videos

Key points

• Current recipes for robot learning require massive amounts of robot demonstrations to obtain robust and adaptive policies.1
• (1) We pretrain a 400M-parameter diffusion policy on our egocentric human dataset, which consists of 100 hours of densely annotated random interactions.1
• This pretraining makes Steertuning robust to background differences and enables spatial and object generalization when deployed on new tasks.1
• (2) Given 5 demonstrations of a new task, we finetune our model for only 2 minutes (2000 training updates) and steer the policy's actions during reverse sampling using the demonstration whose observation is closest to that of the deployment environment.1

Sources (1)

• [1]Steertuning Policies from a Few Human Videos
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 04:55 PM

  In this work, we introduce Steertuning, a system that enables robots to learn new tasks in new environments from 5 human demonstrations in under 5

  Current recipes for robot learning require massive amounts of robot demonstrations to obtain robust and adaptive policies.

Extractive summary: sentences quoted from the sources.