HuMiT: Low-Latency Whole-Body Humanoid Teleoperation via Compact Reference Tracking

Key points

• Existing motion tracking policies typically require dense reference targets including full joint positions, body-part transformations, and even future frames, which tightly couples the controller to expensive upstream modules.1
• Experiments show that our policy achieves a communication-to-inference latency of $\sim$20 ms measured between headset streaming and policy output and a video-based optical flow latency of 65 ms (median) across different motion types.1
• We validate the system on a Unitree G1 humanoid, showcasing split-second reactive responses such as catching a tossed ball, as well as coordinated loco-manipulation capabilities.1

Sources (1)

• [1]HuMiT: Low-Latency Whole-Body Humanoid Teleoperation via Compact Reference Tracking
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 03:45 PM

  Real-time whole-body humanoid teleoperation remains challenging due to the computational latency introduced by optimization-based retargeting pipelines and future-frame conditioning.

  Existing motion tracking policies typically require dense reference targets including full joint positions, body-part transformations, and even future frames, which tightly couples the controller to expensive upstream modules.

Extractive summary: sentences quoted from the sources.