Training in Deployment: Human-in-the-loop Online Post-Training for Robot Manipulation

Key points

• Large pretrained robot foundation models provide strong priors for manipulation, yet adapting them to target tasks with offline teleoperated demonstrations often fails to cover the contact-rich, recovery, and precision-critical states encountered during deployment.1
• Existing methods mitigate this issue by augmenting offline demonstrations with human-in-the-loop data or online reinforcement learning.1
• TIDE introduces a provenance-conditioned supervised learning framework that updates the policy online from human interventions and policy-generated rollouts collected during deployment, without a learned critic, reward model, or action scorer.1
• Across four real-world bimanual manipulation tasks, TIDE improves milestone-normalized task progress and binary full-task completion over the warm-start policy and matched additional offline demonstrations, and performs comparably to the fixed-policy collection baseline while making updated checkpoints available during deployment.1

Sources (1)

• [1]Training in Deployment: Human-in-the-loop Online Post-Training for Robot Manipulation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 06:15 AM

  We present TIDE (Training In DEployment), a human-in-the-loop online post-training system.

  Large pretrained robot foundation models provide strong priors for manipulation, yet adapting them to target tasks with offline teleoperated demonstrations often fails to cover the contact-rich, recovery, and precision-critical states encountered during deployment.

Extractive summary: sentences quoted from the sources.