Rewards for Budget-Aware Multi-Turn Agents That Adapt to Arbitrary Test-Time Constraints
Large language model agents are often given an inference budget and asked to maximise success within it.
ProofPaper ↗
Key points
- A multi-turn agent has to allocate this budget across many environment interactions.
- We ask whether agents can learn this allocation across dynamically specified budgets and which rewards best teach this behaviour.
- We adapt existing length penalties to budget constraints spanning multiple resource types, introduce new reward formulations, and compare eleven objectives in a shared training pipeline.
- Overall, we show that a single agent can learn to allocate resources across turns and adapt its behaviour to user-specified budgets using budget feedback without external budget-allocation controllers.
Sources (1)
- [1]Rewards for Budget-Aware Multi-Turn Agents That Adapt to Arbitrary Test-Time ConstraintsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:49 PM
Large language model agents are often given an inference budget and asked to maximise success within it.
A multi-turn agent has to allocate this budget across many environment interactions.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Voxtral 3
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 20, 2026AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics
- Oct 19, 2026Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
- Oct 19, 2026HarnessRL: Co-Evolving Policies with a Just-in-Time Training Harness for Open-Ended RL
- Oct 18, 2026Bud ContextCompact 2.0: Context Compaction for Agent Success and Accuracy with Small Language Models and Constrained Context Windows
