FM-Occ: Feature-Diverse Upsampling and Memory-Aware Decoding for 3D Occupancy Prediction

Key points

• 3D semantic occupancy prediction estimates voxel-wise occupancy and semantic labels throughout a scene.1
• Our experiments reveal that conventional sparse transposed-convolution upsampling yields poorly differentiated voxel responses and that successive decoder layers discard previously correct semantic predictions.1
• First, 3D Sparse Subvoxel Shuffle (3S) mitigates feature homogenization by constructing candidate child features in the channel dimension, jointly transforming and normalizing them, and then rearranging them onto a finer sparse grid with explicit feature-coordinate correspondence.1
• Second, Entropy-Regulated Layer-wise Memory (ER-LM) retains semantic information across decoder layers by maintaining a layer-wise memory for each query through gated recurrent updates and using prediction entropy to regulate the historical information propagated to the next layer.1

Sources (1)

• [1]FM-Occ: Feature-Diverse Upsampling and Memory-Aware Decoding for 3D Occupancy Prediction
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 08:14 AM

  To address these issues, we present Feature-Diverse Upsampling and Memory-Aware Decoding for 3D Occupancy Prediction (FM-Occ), comprising two components.

  3D semantic occupancy prediction estimates voxel-wise occupancy and semantic labels throughout a scene.

Extractive summary: sentences quoted from the sources.