Investigating Length Bias and Robustness in LVLMs for Multiple-Choice Question Answering

Key points

• We investigate length bias, the systematic preference for longer answer options, through controlled benchmarks spanning fine-grained visual domains.1
• LVLMs consistently favor longer options even when visually inconsistent, with accuracy dropping 2--38% when correct answers are shorter and increasing 1--70% when longer.1
• Through layer-wise mechanistic analysis, we reveal that length bias emerges from late-layer suppression of visual evidence in favor of language-side length priors.1
• We propose and evaluate multiple inference-time mitigation strategies; while these provide partial improvements, none achieves deployable bias mitigation without relying on oracle information unavailable at inference time.1

Sources (1)

• [1]Investigating Length Bias and Robustness in LVLMs for Multiple-Choice Question Answering
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 06:46 AM

  Large Vision-Language Models (LVLMs) demonstrate strong performance on multiple-choice question answering (MCQA), yet their reliability under biased conditions remains underexplored.

  We investigate length bias, the systematic preference for longer answer options, through controlled benchmarks spanning fine-grained visual domains.

Extractive summary: sentences quoted from the sources.