AOV-GS: Active Open-Vocabulary Semantic Mapping with 3D Gaussian Splatting

Key points

• Existing geometry-driven active mapping maximizes coverage but treats semantics as a passive by-product of the collected trajectory, whereas active semantic mapping typically depends on supervised semantic segmentation.1
• We present AOV-GS, an active 3D Gaussian mapping system that guides exploration using frozen foundation models.1
• During coarse exploration, AOV-GS re-ranks geometrically promising candidate views according to CLIP mask-embedding novelty computed from SAM instance segmentation, biasing exploration toward semantically informative regions.1
• Frozen SAM and CLIP provide instance masks and language-aligned embeddings throughout mapping, while a lightweight scene-specific language field is fitted offline directly to the collected CLIP embeddings rather than manually annotated masks, enabling open-vocabulary text-query segmentation on novel views.1

Sources (1)

• [1]AOV-GS: Active Open-Vocabulary Semantic Mapping with 3D Gaussian Splatting
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 08:02 AM

  Robots deployed in human-centered environments need maps that support both geometric navigation and open-vocabulary semantic queries beyond a fixed label set.

  Existing geometry-driven active mapping maximizes coverage but treats semantics as a passive by-product of the collected trajectory, whereas active semantic mapping typically depends on supervised semantic segmentation.

Extractive summary: sentences quoted from the sources.