ResearchResearch paperLarge Language Models · Reinforcement Learning · Training & Scaling1 source · Oct 12, 2026

Efficient Resource-Shiftable RL Post-training with Generative Rewards

We present PivotRL, an RL post-training system that dynamically shifts resources among different roles according to their runtime demands.

Reinforcement learning (RL) post-training with generative reward models (GenRM) has become a central recipe for improving large language models (LLMs), and it is also increasingly resource-intensive: policy rollout, GenRM-based reward evaluation, and policy training all compete for GPU resources.
PivotRL introduces a resource-shiftable runtime built on a Global Parameter Storage, together with a unified mechanism that efficiently shifts role-specific context.
We evaluate PivotRL on a 128-GPU cluster using four model groups spanning dense and mixture-of-experts models.
PivotRL enables resource shifting within seconds and improves end-to-end throughput by 1.81-2.05 times over colocated deployment and 1.50-1.95 times over disaggregated deployment, while preserving RL convergence.

[1]Efficient Resource-Shiftable RL Post-training with Generative Rewards
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 02:33 PM
We present PivotRL, an RL post-training system that dynamically shifts resources among different roles according to their runtime demands.
Reinforcement learning (RL) post-training with generative reward models (GenRM) has become a central recipe for improving large language models (LLMs), and it is also increasingly resource-intensive: policy rollout, GenRM-based reward evaluation, and policy training all compete for GPU resources.

Extractive summary: sentences quoted from the sources.

Before this