X-Splat: Unified Gaussian Splatting for Joint RGB and X-ray 3D Reconstruction

Key points

• We propose a unified representation in which all Gaussian primitives share a single set of geometric parameters, with only surface-proximal primitives additionally carrying appearance attributes.1
• We perform cross-modal registration through clean multi-view stereo and X-ray geometric proxies instead of aligning two sets of optimized Gaussians, improving alignment stability.1
• We then jointly optimize the unified representation via supervision from both modalities, complemented by a surface-depth consistency loss derived from X-ray geometry to guide RGB reconstruction.1
• Experiments on the InsideOut dataset and our Flower & Insect dataset show that our framework achieves more robust cross-modal alignment, higher RGB rendering quality without compromising X-ray reconstruction, and a more compact representation than maintaining two independent geometries.1

Sources (1)

• [1]X-Splat: Unified Gaussian Splatting for Joint RGB and X-ray 3D Reconstruction
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 10:06 AM

  Jointly reconstructing RGB and X-ray provides new insights by connecting external appearance with internal structure, enabling a more complete understanding of real-world objects than single modality.

  We propose a unified representation in which all Gaussian primitives share a single set of geometric parameters, with only surface-proximal primitives additionally carrying appearance attributes.

Extractive summary: sentences quoted from the sources.