RoboPrompt: Intuitive Robot Policy Steering with Sparse Human Input

Key points

• End-to-end robot policies trained through imitation learning remain constrained by limited data diversity, making reliable zero-shot deployment in real-world settings challenging.1
• Shared-autonomy methods enable human correction through teleoperation, but specialized hardware and operator training hinder deployment at scale.1
• RoboPrompt decouples human-intention translation from the underlying policy: a reusable module converts human guidance into action drafts, which are refined through the diffusion or flow-matching dynamics of the base policy.1
• By controlling action generation in noise space, RoboPrompt balances human intent with the policy prior without modifying the base policy architecture or fine-tuning it for steerability.1

Sources (1)

• [1]RoboPrompt: Intuitive Robot Policy Steering with Sparse Human Input
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 05:58 PM

  We present RoboPrompt, a general-purpose, lightweight robot policy steering system that enables users to guide policy behavior through intuitive, sparse inputs, including drawn traces, target points, and coarse directional instructions.

  End-to-end robot policies trained through imitation learning remain constrained by limited data diversity, making reliable zero-shot deployment in real-world settings challenging.

Extractive summary: sentences quoted from the sources.