Directionally Factorized Illumination with Spherical Harmonic Rotation

Key points

• Many real-world capture scenarios involve an active light source rigidly attached to the camera: camera flashes for dark-environment imaging, underwater vehicle lights, and endoscopic/laparoscopic medical procedures.1
• Neural rendering methods (NeRF, Gaussian Splatting) encode this pose-dependent illumination as surface coloration, producing appearance inconsistencies and making it harder to recover the underlying scene.1
• It factorizes per-point emitted color into surface albedo and a spherical harmonic (SH)-represented illumination factor.1
• DaFISH provides analytical generalization of illumination to arbitrary camera poses without retraining.1

Sources (1)

• [1]Directionally Factorized Illumination with Spherical Harmonic Rotation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 06:35 PM

  We present DaFISH, which can be plugged into any novel-view synthesis (NVS) pipeline such as NeRF.

  Many real-world capture scenarios involve an active light source rigidly attached to the camera: camera flashes for dark-environment imaging, underwater vehicle lights, and endoscopic/laparoscopic medical procedures.

Extractive summary: sentences quoted from the sources.