Learning Adaptable Dexterous Skills via Hybrid Dexterous Mimicking

Key points

• Dexterous multi-finger hands hold promise to unlock diverse skills like functional grasps, in-hand reorientation, and dynamic handover that the human-designed world assumes, yet existing learning recipes force a trade-off.1
• Goal-reaching reinforcement learning generalizes to arbitrary object poses but, lacking any human prior, converges to unnatural and unreliable finger strategies; motion tracking reproduces human-like behavior from a single clip but cannot deviate from it.1
• From seconds of human motion per skill, the resulting policies can learn diverse adaptable dexterous skills, such as functional grasping, object reposing, in-hand reorientation with finger gaiting, and dynamic behaviors like throw-and-catch.1
• Our method outperforms baselines in terms of generalization and functionality, and we demonstrate transfer across simulators and to the real world.1

Sources (1)

• [1]Learning Adaptable Dexterous Skills via Hybrid Dexterous Mimicking
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:59 PM

  We present AdaDex, a goal-conditioned RL framework that turns a single hand-object trajectory of a few seconds---from mocap data or HOI reconstruction from in-the-wild RGB(D) video---into an adaptable dexterous skill.

  Dexterous multi-finger hands hold promise to unlock diverse skills like functional grasps, in-hand reorientation, and dynamic handover that the human-designed world assumes, yet existing learning recipes force a trade-off.

Extractive summary: sentences quoted from the sources.