OmniPoser: Flexible Human Motion Recovery in the Wild with Masked Flow Matching

Key points

• Recovering 3D human motion from sparse wearable sensors is challenging due to the inherent noise, occlusion, and instability of real-world tracking signals.1
• OmniPoser introduces a cross-modal masking mechanism paired with a Masked Conditioning Encoder, enabling one architecture to incorporate diverse sensor configurations through a masked design.1
• A decoupled dual-stream generative backbone generates complete motion via Conditional Flow Matching, requiring only a single ODE integration step at inference.1
• Across three settings and seven benchmarks, including in-the-wild Nymeria and Ego-Exo4D datasets, OmniPoser performs on par with state-of-the-art methods while running at over 700 FPS.1

Sources (1)

• [1]OmniPoser: Flexible Human Motion Recovery in the Wild with Masked Flow Matching
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 09:22 AM

  We propose OmniPoser, a universal framework that recovers full-body SMPL motion from a flexible combination of heterogeneous inputs, such as 3D keypoints, sparse IMU rotations, head-mounted 6-DoF poses, or any subset thereof, within a single model.

  Recovering 3D human motion from sparse wearable sensors is challenging due to the inherent noise, occlusion, and instability of real-world tracking signals.

Extractive summary: sentences quoted from the sources.