Training Generalist Navigation Policies via Action Self-Distillation

Key points

• Language provides a natural interface for humans to specify and reason about goals, but training language-conditioned robotic policies and reasoning typically requires expensive manual annotations.1
• During training, the self-teacher receives privileged trajectory feedback and infers the underlying navigation goal together with goal-oriented reasoning.1
• The student is then trained to condition on the inferred navigation goal, predict future ego-frame trajectory, and provide a description of the scene semantics, plan, and reasoning, from onboard observations alone.1
• Across 13 challenging indoor and outdoor real-world navigation tasks, our method improves success rate over the strongest baseline by $30%$ in indoor settings, $30%$ on multi-stage tasks, and achieves $95%$ success rate on outdoor tasks, validating that privileged trajectory feedback can enable self-distillation for scalable generalist navigation policies without human annotations.1

Sources (1)

• [1]Training Generalist Navigation Policies via Action Self-Distillation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 02:11 AM

  We propose Action Self-Distillation (ASD), a method for learning language-conditioned reasoning for robotic navigation without human-annotated language or reasoning labels.

  Language provides a natural interface for humans to specify and reason about goals, but training language-conditioned robotic policies and reasoning typically requires expensive manual annotations.

Extractive summary: sentences quoted from the sources.