CoGaussian: Open-Vocabulary Gaussian-based Collaborative 3D Scene Understanding

Key points

• Collaborative perception enables agents to retrieve the scene semantics and geometry beyond the field of view of any individual agent.1
• However, existing collaborative semantic occupancy methods rely on dense 3D voxel annotations and are limited to recognizing objects within a predefined set of semantic classes.1
• CoGaussian represents a scene as a set of foundation model-aligned sparse 3D Gaussians, which serve as a compact representation for communicating and fusing geometric and semantic information across multiple agents.1
• On the Semantic-OPV2V benchmark, our method outperforms existing open-vocabulary single-agent approaches, highlighting the effectiveness of open-vocabulary collaborative scene understanding.1

Sources (1)

• [1]CoGaussian: Open-Vocabulary Gaussian-based Collaborative 3D Scene Understanding
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:04 PM

  To overcome these limitations, we present CoGaussian, an open-vocabulary Gaussian-based collaborative semantic occupancy prediction framework.

  Collaborative perception enables agents to retrieve the scene semantics and geometry beyond the field of view of any individual agent.

Extractive summary: sentences quoted from the sources.