Auditing Where Concept Bottleneck Accuracy Comes From on CUB

Key points

• We audit where the accuracy of CBMs and Concept Embedding Models (CEMs) on CUB comes from.1
• The CUB annotations are too noisy to explain it: class-level training labels agree with per-image annotations on only 82% of image--concept pairs, and classifiers trained on per-image annotations reach only 37% species accuracy.1
• We identify four design axes through which class information can enter concept scores---receptive field, shared versus concept-specific representations, norm-dependent scoring, and task supervision.1
• Localization and occlusion show that locality makes grounding measurable but does not guarantee it: under class-level labels, local concept scores use the head as a proxy for body, wing, and tail concepts, and CEM concept scores barely respond to occluding their supporting part.1

Sources (1)

• [1]Auditing Where Concept Bottleneck Accuracy Comes From on CUB
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 11:47 AM

  Concept Bottleneck Models (CBMs) promise predictions that humans can inspect and correct through concepts such as "curved beak".

  We audit where the accuracy of CBMs and Concept Embedding Models (CEMs) on CUB comes from.

Extractive summary: sentences quoted from the sources.