Perception or Reasoning? Diagnosing and Tackling Reinforcement Learning Bottlenecks in VLMs

Key points

• Reinforcement learning has brought strong gains to language model reasoning, but its improvements on visual tasks remain less consistent.1
• One difficulty is that visual task failures can arise from different sources: the same benchmark may be limited by perception for one model and by reasoning for another.1
• More importantly, it shows that RL gains become smaller as perceptual headroom increases, but larger when reasoning explains more of the recoverable errors, suggesting that current RL exploits reasoning headroom more effectively than perceptual headroom.1
• Motivated by this gap, we analyze how existing RL objectives affect visual perception and find that reward- and token-level signals influence visual attention only indirectly.1

Sources (1)

• [1]Perception or Reasoning? Diagnosing and Tackling Reinforcement Learning Bottlenecks in VLMs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 07:00 AM

  We introduce Perception and Reasoning Attribution (PARA), which measures the remaining perceptual and reasoning headroom of each model--benchmark pair by comparing how many current errors are recovered with more accurate visual information versus a stronger reasoner.

  Reinforcement learning has brought strong gains to language model reasoning, but its improvements on visual tasks remain less consistent.

Extractive summary: sentences quoted from the sources.