HORIA: 4D Human and Object Reconstruction from a Monocular Video with Interaction-Aware Re-Rendering

Key points

• To address these challenges, we introduce HORIA, built upon two core designs.1
• Second, conditioned on these text descriptions of the interaction, we propose an interaction-aware re-rendering that compensates for the limited single-view observation.1
• Specifically, we render the initially reconstructed 4D human and object from novel viewpoints and refine the renderings with a text-to-video diffusion prior, which provides auxiliary novel-view supervision beyond single-view cues.1
• As a result, our framework produces temporally coherent and accurate 4D reconstructions under diverse interaction scenarios, achieving state-of-the-art performance.1

Sources (1)

• [1]HORIA: 4D Human and Object Reconstruction from a Monocular Video with Interaction-Aware Re-Rendering
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 07:52 AM

  Joint 4D reconstruction of a human and object from a monocular video is essential for understanding how people interact with their environment.

  To address these challenges, we introduce HORIA, built upon two core designs.

Extractive summary: sentences quoted from the sources.