Where Should a Stuck Agent Spend Its Next Tokens? Causal Attribution of Inference-Time Resources

Key points

• Inference-time scaling is often treated as a token-budget problem, but a failed coding agent can seek qualitatively different resources: computation, information, or feedback.1
• We evaluate six models on 194 function-level failures from MBPP+ and HumanEval+ and 126 repository-level failures from SWE-bench Verified.1
• Task-relevant specifications consistently improve repair, whereas public-test feedback provides small gains and retrieved code little to none.1
• The value of additional computation declines with model capability, while information remains useful; resource channels largely exhibit diminishing returns, although complementary information can reinforce itself.1

Sources (1)

• [1]Where Should a Stuck Agent Spend Its Next Tokens? Causal Attribution of Inference-Time Resources
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 06:59 PM

  We introduce Banzhaf Resource Attribution (BRA), a controlled factorial framework that measures each resource's marginal contribution to repair and its interactions while holding the failed state and revision protocol fixed.

  Inference-time scaling is often treated as a token-budget problem, but a failed coding agent can seek qualitatively different resources: computation, information, or feedback.

Extractive summary: sentences quoted from the sources.