Adaptive Decoding Methods for Token-Efficient Reinforcement Learning
Rollout decoding affects both which trajectories are used for reinforcement learning with verifiable rewards (RLVR) and how many completion tokens are spent collecting them.
ProofPaper ↗
Key points
- We study whether candidate-token support should remain fixed or widen during training in the context of rollout sampling efficiency.
- In a 4B code-RLVR setting, the schedule has a higher interpolated 150M-token validation value than its matched fixed-K=10 continuation in all three runs, with a mean gain of $1.80{\pm}0.42$ points.
- An exploratory single-run test replay yields identical point estimates for the schedule and fixed $K=10$.
- Overall, our results show that rollout support is an important control for sampling efficiency in RLVR and motivate further study of adaptive support schedules.
Sources (1)
- [1]Adaptive Decoding Methods for Token-Efficient Reinforcement LearningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 06:13 AM
Rollout decoding affects both which trajectories are used for reinforcement learning with verifiable rewards (RLVR) and how many completion tokens are spent collecting them.
We study whether candidate-token support should remain fixed or widen during training in the context of rollout sampling efficiency.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026HarnessRL: Co-Evolving Policies with a Just-in-Time Training Harness for Open-Ended RL
- Oct 18, 2026RARE: Reinforcement Learning with Adaptive Rubric Evolution for Open-Ended Generation
- Oct 18, 2026Conformal Prediction-Enabled Sequence-level Policy Optimization for Reasoning Language Models
- Oct 17, 2026ProbePO: Sparse Probes, Redistributed Credit for Critic-Free LLM Reinforcement Learning
- Oct 17, 2026Axiomatic World Modeling for Physics Reasoning
- Oct 16, 2026Training-Free Prompt Selection for Efficient Reinforcement Learning With Partial Rollouts