TILDE: Teacher-in-the-Loop Distillation from Embodied Foundation Models

Key points

• Large-scale action pretraining equips Embodied Foundation Models (EFMs) with strong manipulation capabilities, with particular strengths in generalization and closed-loop robustness, but reproducing such pretraining requires substantial robot data and computation.1
• We further propose AutoDAgger, an autonomous data aggregation pipeline in which a robotic reward model is utilized to monitor task progress and trigger teacher intervention, thereby collecting recovery experience without human monitoring or manual control.1
• AutoDAgger accelerates recovery-trajectory collection by more than $2\times$ compared with the human-in-the-loop method.1
• By unifying learning from demonstrations, on-policy teacher supervision, and teacher's corrective behavior, TILDE improves task performance by up to $4.1\times$ over behavior cloning across three real-world tasks and supports effective cross-architecture distillation.1

Sources (1)

• [1]TILDE: Teacher-in-the-Loop Distillation from Embodied Foundation Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 04:26 PM

  To transfer these capabilities from pretrained EFMs to policies without action pretraining, we introduce TILDE (Teacher-In-the-Loop Distillation from EFMs), which synergizes on-policy distillation at student-induced states with learning from teacher-executed recovery trajectories.

  Large-scale action pretraining equips Embodied Foundation Models (EFMs) with strong manipulation capabilities, with particular strengths in generalization and closed-loop robustness, but reproducing such pretraining requires substantial robot data and computation.

Extractive summary: sentences quoted from the sources.