EmbodyHOI: Embodiment-Aware Generation of Hand-Object Interactions for Humanoid Dexterous Manipulation

Key points

• Humanoid dexterous manipulation needs reference motions that are both semantically rich and kinematically feasible for the target robot.1
• Text-conditioned hand-object interaction (HOI) generators supply diverse, language-specified demonstrations, but they synthesize motion in human (MANO) space.1
• We feed back gradients from the embodiment-aware IK solver into the HOI sampler and jointly optimize the position of the humanoid with respect to the object.1
• On ARCTIC trajectories with two contrasting humanoids (Unitree G1 and H1-2), our proposed guidance-based method cuts the retargeted wrist error from 8.9 cm to 0.6 cm while leaving text-to-HOI generation quality intact, and raises task success from 68% to 79% on 50 random G1 tasks and from 62% to 76% on a four-task H1-2 suite.1

Sources (1)

• [1]EmbodyHOI: Embodiment-Aware Generation of Hand-Object Interactions for Humanoid Dexterous Manipulation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 08:08 PM

  We present EmbodyHOI, which runs a differentiable inverse-kinematics (IK) solver for a specific humanoid embodiment inside the sampling loop of a frozen flow-matching HOI trajectory generator.

  Humanoid dexterous manipulation needs reference motions that are both semantically rich and kinematically feasible for the target robot.

Extractive summary: sentences quoted from the sources.