MYOPIA: Understanding the Limits of Tree-Based Reasoning in Vision-Language Models

Key points

• Decision trees can make vision-language model (VLM) classification interpretable by replacing a single prediction with a sequence of root-to-leaf questions.1
• Across GPT-4o, Qwen-VL Max, and LLaMA-3.2 11B Vision, tree traversal underperforms zero-shot prompting in all 12 dataset--model pairs; paired bootstrap analyses consistently favor zero-shot.1
• Incorrect local decisions then irreversibly exclude the correct class, while answer extraction introduces additional model-dependent loss.1
• MYOPIA therefore provides a framework for diagnosing hierarchical visual classification and motivates testing tree designs that prioritize perceptual answerability and recovery from early errors.1

Sources (1)

• [1]MYOPIA: Understanding the Limits of Tree-Based Reasoning in Vision-Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 09:25 AM

  We introduce MYOPIA, a benchmark comprising four LLM-generated, human-refined decision trees for GTSRB, MNIST, CIFAR-10, and MSCOCO, together with matched prompts, diagnostic protocols, and an evaluation harness.

  Decision trees can make vision-language model (VLM) classification interpretable by replacing a single prediction with a sequence of root-to-leaf questions.

Extractive summary: sentences quoted from the sources.