Self-Supervised Learning of Semantically Consistent Visual Features via a Simple 3D Prior

Key points

• However, such features exhibit documented limitations that hinder accurate correspondence estimation such as the inability to disambiguate the left from right for categories exhibiting symmetries or repeated parts.1
• Previous attempts to improve these features rely on using supervised data such as annotated keypoint pairs or camera pose information, or make restrictive topological assumptions, such as modelling the object of interest as a 3D sphere.1
• We introduce a new self-supervised approach that allows us to learn features for topologically complex object categories using a simple prior.1
• Instead of relying on an expensive instance-level analysis by synthesis pipeline, our approach makes use of contrastive learning and distribution matching at the global dataset-level to learn the coarse shape and appearance of a category.1

Sources (1)

• [1]Self-Supervised Learning of Semantically Consistent Visual Features via a Simple 3D Prior
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 03:42 PM

  Visual features extracted from vision models like DINOv2 or Stable Diffusion enable high quality semantic correspondence between images.

  However, such features exhibit documented limitations that hinder accurate correspondence estimation such as the inability to disambiguate the left from right for categories exhibiting symmetries or repeated parts.

Extractive summary: sentences quoted from the sources.