Diffeomorphic Hidden-State Steering: Improving Imitation Learning Policy Robustness through Expert Hidden-State Geometry

Key points

• Small observation or action errors can compound during closed-loop execution and disrupt a trained imitation learning policy.1
• We study whether the original expert demonstrations can improve robustness through test-time hidden-state steering without additional demonstrations or policy retraining.1
• Our analysis of a trained policy finds that expert hidden-state variation is concentrated in relatively few dimensions, while the directions capturing this variation differ across regions.1
• Combining DHS with action smoothing improves mean success rate under observation noise in certain tasks compared with action smoothing alone.1

Sources (1)

• [1]Diffeomorphic Hidden-State Steering: Improving Imitation Learning Policy Robustness through Expert Hidden-State Geometry
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 10:48 AM

  Motivated by this geometry, we propose Diffeomorphic Hidden-State Steering (DHS), which learns an invertible nonlinear coordinate map from expert hidden states obtained by replaying the demonstrations through the frozen policy.

  Small observation or action errors can compound during closed-loop execution and disrupt a trained imitation learning policy.

Extractive summary: sentences quoted from the sources.