BOSS: Token-Budgeted Online Sampling and Selection for LLM Reasoning

Key points

• We study within-instance allocation: under a hard per-instance prompt-plus-completion token cap, when should inference stop, generate another candidate, or spend residual tokens adjudicating candidates already observed?1
• Across two Llama-3.1 models and four reasoning benchmarks, BOSS attains the highest accuracy in 22 of 24 model-task-budget cells under the common hard cap.1
• Approximately matched-cost controls further characterize how the benefit of adjudication changes with available inference budget.1
• Together, these results establish generation-adjudication allocation as a distinct design axis for reasoning under strict per-instance caps.1

Sources (1)

• [1]BOSS: Token-Budgeted Online Sampling and Selection for LLM Reasoning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 01:34 PM

  Test-time scaling can improve LLM reasoning, but existing approaches typically adapt generation or apply answer selection after a predetermined generation schedule.

  We study within-instance allocation: under a hard per-instance prompt-plus-completion token cap, when should inference stop, generate another candidate, or spend residual tokens adjudicating candidates already observed?

Extractive summary: sentences quoted from the sources.