Towards Reconstructing Geographically Diverse Architecture with 3D Foundation Models

Key points

• Internet-scale photo collections of real-world landmarks have driven progress in 3D computer vision yet remain highly challenging for modern 3D foundation models (3DFMs) that estimate scene structure in a single feed-forward pass.1
• We examine socially relevant disparities in model error and find that model performance varies by geographic region; however, unlike typical fairness regimes, the trends only partially match training data representation.1
• Our investigation reveals that the observed geographic disparities reflect not only differences in data representation and scene size but also differences in region-specific architectural features and background distractors that make scenes from different regions inherently vary in difficulty.1
• Leveraging insights from these model failings, we introduce an intervention scheme that improves 3DFM performance while simultaneously reducing geographic disparities.1

Sources (1)

• [1]Towards Reconstructing Geographically Diverse Architecture with 3D Foundation Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 06:08 PM

  In this work, we introduce ArchWorld, a benchmark of 3D architectural landmarks with high geographic coverage and rich metadata.

  Internet-scale photo collections of real-world landmarks have driven progress in 3D computer vision yet remain highly challenging for modern 3D foundation models (3DFMs) that estimate scene structure in a single feed-forward pass.

Extractive summary: sentences quoted from the sources.