Sample, Mix, or Verify? Allocating Test-Time Compute under Imperfect Verifiers

Key points

• The first grows predictably with the number of samples; the second does not, because real verifiers (unit tests, majority votes, LLM judges, reward models) accept wrong answers, reject right ones, and make correlated mistakes.1
• With a fixed budget, a practitioner must decide how many samples to draw, which models to draw them from, and how much to spend on verification, and today these choices are made by rules of thumb.1
• We introduce SPLIT, which makes them from a small labelled pilot.1
• Across seven benchmarks spanning math, science QA, code and software-engineering agents, 26 models, 175 agent systems and six kinds of real verifiers, all from public releases, SPLIT lands within 0.02-0.8 percentage points of the best allocation in hindsight, while every fixed rule we test loses at least 2.5 points in some setting.1

Sources (1)

• [1]Sample, Mix, or Verify? Allocating Test-Time Compute under Imperfect Verifiers
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 10:28 AM

  A common way to make language models and agents more accurate is to spend more compute at inference time: generate many candidate solutions and submit the one a verifier prefers.

  The first grows predictably with the number of samples; the second does not, because real verifiers (unit tests, majority votes, LLM judges, reward models) accept wrong answers, reject right ones, and make correlated mistakes.

Extractive summary: sentences quoted from the sources.