Measure First, Then Allocate: Calibrated Rollout Budgets for MCP Agents in Reinforcement Learning

Key points

• Training tool-using language models requires costly interaction, yet identical rewards within a rollout group provide no group-relative learning signal.1
• We first introduce Discord, which tests harness edits using exact paired inference and decision-preserving early stopping: only disagreeing paired outcomes favor one harness over another.1
• This insight motivates prospective allocation in BRACE (Bank-calibrated Rollout Allocation under Correlated Episodes).1
• BRACE combines historical task evidence, uncertainty in task success rates, and a correlated reward model to forecast mixed-reward groups before allocating generation budget.1

Sources (1)

• [1]Measure First, Then Allocate: Calibrated Rollout Budgets for MCP Agents in Reinforcement Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 06:15 PM

  We use reward disagreement to guide harness evaluation and the allocation of rollouts for policy training in Model Context Protocol (MCP) environments.

  Training tool-using language models requires costly interaction, yet identical rewards within a rollout group provide no group-relative learning signal.

Extractive summary: sentences quoted from the sources.