EgoRayFormer: Ray-Space Stereo Refinement for Egocentric 3D Human Pose Estimation

Key points

• Head-mounted stereo fisheye cameras provide a compact view of the wearer's body, but strong distortion, partial visibility and a short baseline make 3D pose estimation difficult.1
• Existing refinement decoders sample image features independently in each view, without requiring the samples to correspond to a common 3D hypothesis.1
• This formulation introduces an explicit stereo constraint while retaining flexibility when the rays provide weak depth constraints.1
• In local evaluations on the official device-relative test splits, EgoRayFormer achieves three-seed mean MPJPEs of 17.64 mm on UnrealEgo2 and 49.55 mm on UnrealEgo-RW, improving to 47.57 mm with UnrealEgo2 pre-training.1

Sources (1)

• [1]EgoRayFormer: Ray-Space Stereo Refinement for Egocentric 3D Human Pose Estimation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 04:03 PM

  We introduce EgoRayFormer, a causal pose estimator that couples stereo feature sampling and geometric refinement in ray space.

  Head-mounted stereo fisheye cameras provide a compact view of the wearer's body, but strong distortion, partial visibility and a short baseline make 3D pose estimation difficult.

Extractive summary: sentences quoted from the sources.