Frustratingly Easy Majority Voting Using Length Conditioning

Key points

• Majority Voting (MV) is a simple and widely used strategy which improves the reasoning accuracy of Large Language Models (LLMs) by allocating additional inference-time computation.1
• Specifically, in MV multiple Chain-of-Thoughts (CoTs) are independently sampled and the most frequent final answer is selected.1
• We formulate this inductive bias as a length-conditioned prior probability, which is modeled by a simple logistic function.1
• Extensive experiments across multiple LLMs and reasoning benchmarks show that LCMV consistently improves over both standard MV and existing length-based and confidence-based aggregation strategies, while retaining the simplicity and model-agnostic nature of MV.1

Sources (1)

• [1]Frustratingly Easy Majority Voting Using Length Conditioning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 09:48 AM

  In this work, we propose Length-Conditioned Majority Voting (LCMV), a Test-Time Training approach which improves MV leveraging an empirical observation that has recently been established in several studies: longer CoTs tend to be associated with a lower probability of correctness.

  Majority Voting (MV) is a simple and widely used strategy which improves the reasoning accuracy of Large Language Models (LLMs) by allocating additional inference-time computation.

Extractive summary: sentences quoted from the sources.