VisibilitySplat: Ray-Traced Occlusion for Relighting with 3D Gaussian Splatting

Key points

• Our approach augments the standard rasterized 3DGS pipeline with a sparse visibility oracle that estimates transmittance-weighted occlusion for Gaussian primitives from sampled lighting directions.1
• This oracle is implemented using a point-based ray tracer accelerated by a bounding volume hierarchy (BVH) and queried sparsely within the inverse-rendering objectives, while the original 3DGS loop is retained for training efficiency, interactive visualization, and tooling compatibility.1
• Shading follows a physically based rendering (PBR) formulation with a principled microfacet bidirectional reflectance distribution function (BRDF) and low-resolution high-dynamic-range (HDR) environment maps.1
• Overall, the findings suggest that sparse, point-based visibility queries effectively complement rasterized splatting for inverse rendering while preserving the practical advantages of the 3DGS ecosystem.1

Sources (1)

• [1]VisibilitySplat: Ray-Traced Occlusion for Relighting with 3D Gaussian Splatting
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 09:17 AM

  We address inverse rendering with 3D Gaussian Splatting (3DGS) by introducing explicit visibility modeling for relighting as well as material and light decomposition.

  Our approach augments the standard rasterized 3DGS pipeline with a sparse visibility oracle that estimates transmittance-weighted occlusion for Gaussian primitives from sampled lighting directions.

Extractive summary: sentences quoted from the sources.