Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration

Key points

• While learning dexterous manipulation from a single human video offers a promising alternative to costly robot demonstrations, many recent methods predominantly imitate demonstrated motions.1
• Such strict motion matching often limits generalization to initial object poses, goal poses, and grasps not shown in the video.1
• Our key insight is to abstract the video into sequential scene graphs that guide RL, enabling efficient exploration while preserving broad generalizability.1
• Trained entirely in simulation, Dex-One2Many transfers zero-shot to a real multi-fingered hand.1

Sources (2)

• [1]Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:59 PM

  To address these coupled generalization and exploration challenges, we present Dex-One2Many, a real-to-sim-to-real framework that learns a generalizable dexterous manipulation policy from a single human video.

  While learning dexterous manipulation from a single human video offers a promising alternative to costly robot demonstrations, many recent methods predominantly imitate demonstrated motions.

• [2]Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration
  Hugging Face Daily Papers · Oct 8, 05:59 PM · same content

Extractive summary: sentences quoted from the sources.