From RL Papers to Framework Code: Inclusion and Review in Seven Open-Source Training Frameworks
Practitioners use reinforcement learning methods for language models through training frameworks, so a method that a framework includes is one configuration change away for its users.
ProofPaper ↗
Key points
- We trace seven open-source frameworks for RL post-training through their commit histories and link them to 6,939 screened arXiv papers from January 2025 to June 2026.
- Of 1,598 papers that propose a general RL training method, 37 reached at least one framework.
- Both gaps hold without papers from the companies that maintain the frameworks, while code release was equally common in both groups.
- We release the data as a tracker that links training failures to framework implementations and lists each framework's contribution rules and earlier proposals.
Sources (2)
- [1]From RL Papers to Framework Code: Inclusion and Review in Seven Open-Source Training FrameworksHugging Face Daily Papers · Oct 20, 12:00 AM
Practitioners use reinforcement learning methods for language models through training frameworks, so a method that a framework includes is one configuration change away for its users.
We trace seven open-source frameworks for RL post-training through their commit histories and link them to 6,939 screened arXiv papers from January 2025 to June 2026.
- [2]From RL Papers to Framework Code: Inclusion and Review in Seven Open-Source Training FrameworksarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 11:16 AM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Look Before You Reason: Implicit Visual Thinking for Efficient Multimodal Reasoning
- Oct 19, 2026Mitigating Reward Hacking in Preference Reward Models with Evolving Penalties
- Oct 18, 2026TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting [R]
- Oct 18, 2026Explore or Confirm? LLM Agents Misallocate Measurements Under Noise
- Oct 18, 2026Google Open-Sources Harness2, a Recursive Agent Harnessing Framework
- Oct 18, 2026Conformal Prediction-Enabled Sequence-level Policy Optimization for Reasoning Language Models