DexOAK: Object-Aware Kinematic Retargeting

Key points

• DexOAK first minimizes an interaction-graph Laplacian objective over a joint hand-object graph, so the robot reproduces the demonstrated object-relative interaction geometry rather than the human hand pose alone.1
• These per-frame solutions form seed retargeted references, from which a feed-forward network conditioned on human keypoints and object point clouds learns the retargeting function.1
• Object-aware finetuning then refines the network under contact, non-penetration, and self-collision losses that position the fingers on the demonstrated contacts and reduce artifacts destabilizing physics-based refinement.1
• DexOAK references improve dynamic retargeting over the state-of-the-art across bimanual tasks with rigid and articulated objects.1

Sources (1)

• [1]DexOAK: Object-Aware Kinematic Retargeting
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 05:20 PM

  We introduce DexOAK, a learning-based object-aware kinematic retargeting framework for generating contact-consistent robot hand references from human motion-capture data.

  DexOAK first minimizes an interaction-graph Laplacian objective over a joint hand-object graph, so the robot reproduces the demonstrated object-relative interaction geometry rather than the human hand pose alone.

Extractive summary: sentences quoted from the sources.