RealMe: Immersive Volumetric Video from Minimal Captures

Key points

• This paper addresses the problem of creating immersive volumetric videos from sparse and low-resolution capture resources.1
• Conventional dense multi-view reconstruction methods can produce high-quality volumetric videos, but their reliance on costly capture setups makes them impractical in real-world scenarios.1
• The video diffusion model adapts factorized spatio-temporal attention to the multi-view capture layout, denoises these latents across viewpoints and time, and decodes them into enhanced videos to refine the 4D representation.1
• Extensive experiments on the public DNA-Rendering benchmark and our self-captured RealVV dataset demonstrate that RealMe is able to create immersive volumetric videos and outperforms state-of-the-art methods in visual fidelity and 4D consistency.1

Sources (2)

• [1]RealMe: Immersive Volumetric Video from Minimal Captures
  Hugging Face Daily Papers · Oct 21, 12:00 AM

  To overcome these constraints, we propose RealMe, a unified framework built around a high-resolution video diffusion model that enhances limited captures into high-resolution (2K), 4D-consistent videos for volumetric video reconstruction.

  This paper addresses the problem of creating immersive volumetric videos from sparse and low-resolution capture resources.

• [2]RealMe: Immersive Volumetric Video from Minimal Captures
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 04:49 PM

Extractive summary: sentences quoted from the sources.