Where Small Mistakes Matter: Quantifying State Criticality for Imitation Learning

Key points

• Imitation learning typically treats every demonstration state as equally important, optimizing a behavior-cloning objective uniformly across the dataset.1
• We then propose SCRIM, a practical State CRIticality Measure that estimates this quantity by training an adversarial disturbance policy and a value critic around a fixed imitation policy.1
• We show that these scores can be used to reweight behavior-cloning losses, improving imitation at consequential states and increasing overall task performance.1
• We evaluate SCRIM across various simulated and real-world robotic manipulation tasks, and demonstrate that SCRIM improves policy training, enables efficient data selection, and provides interpretable insights into which moments in a demonstration are most critical for task success.1

Sources (1)

• [1]Where Small Mistakes Matter: Quantifying State Criticality for Imitation Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 04:37 AM

  We formalize state criticality in imitation learning as the worst-case degradation in downstream value induced by a bounded perturbation of the expert action at a given state.

  Imitation learning typically treats every demonstration state as equally important, optimizing a behavior-cloning objective uniformly across the dataset.

Extractive summary: sentences quoted from the sources.