AWLS: Advantage-Weighted Latent Steering for One-Step Generative Robotic Manipulation Policies

Key points

• Human-collected real-world robotic manipulation demonstrations often contain mixed-quality trajectories, including hesitant, indirect, or inefficient action segments.1
• Offline RL value estimates offer a natural signal for biasing policies toward higher-value behavioral modes, but directly fine-tuning the generative policy or using critic gradients for policy steering can amplify critic errors and induce prior drift.1
• We further introduce a rollout-free decoder-sensitivity criterion for selecting pretrained checkpoints that remain responsive to latent-noise perturbations.1
• Experiments on simulated and real-world robotic manipulation tasks with clean and mixed-quality demonstrations show that AWLS improves over the frozen base policy while limiting drift from the pretrained behavior prior.1

Sources (1)

• [1]AWLS: Advantage-Weighted Latent Steering for One-Step Generative Robotic Manipulation Policies
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 06:58 AM

  We propose Advantage-Weighted Latent Steering (AWLS), an offline post-training method for improving frozen one-step generative manipulation policies without updating their decoders.

  Human-collected real-world robotic manipulation demonstrations often contain mixed-quality trajectories, including hesitant, indirect, or inefficient action segments.

Extractive summary: sentences quoted from the sources.