Smoothness Ranks Skill, Not Success: An Audit of Published Trajectory-Smoothness Curation Metrics Under Episode-Length Controls
Trajectory smoothness has been proposed as a cheap, training-free signal for curating imitation-learning demonstrations, most recently by rinse, which introduces Spectral Arc Length (SAL) and a contact-aware Trajectory-Envelope Distance (TED).
ProofPaper ↗
Key points
- We test whether these published metrics survive episode-length controls on real robomimic data.
- On can/mh, SAL separates better from worse operators with AUROC 0.952, but episode length alone achieves 0.957, and inside a duration-matched band SAL falls to 0.628 while raw mean jerk, the baseline rinse argues against, rises to 0.896.
- Training GMM behavior-cloning policies on each metric's curated subset (10 seeds), curating by TED costs roughly a third of downstream success relative to random selection of the same size (0.240 vs. 0.360, p=0.0001, surviving Bonferroni correction), while SAL and jerk are indistinguishable from not curating.
- Inspecting trajectories shows why TED inverts: successful manipulation requires sustained high-deviation motion through contact, while failure is low-amplitude hovering that a smoothed envelope follows closely.
Sources (1)
- [1]Smoothness Ranks Skill, Not Success: An Audit of Published Trajectory-Smoothness Curation Metrics Under Episode-Length ControlsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 09:39 PM
Trajectory smoothness has been proposed as a cheap, training-free signal for curating imitation-learning demonstrations, most recently by rinse, which introduces Spectral Arc Length (SAL) and a contact-aware Trajectory-Envelope Distance (TED).
We test whether these published metrics survive episode-length controls on real robomimic data.
Extractive summary: sentences quoted from the sources.