What RL Brings to Reasoning: A Few Decisions the Base Model Would Not Make
Reinforcement learning with verifiable rewards (RLVR) substantially improves mathematical reasoning, but what it adds to the base model remains unclear.
ProofPaper ↗
Key points
- We investigate this through a sequence of generation-time interventions on Qwen3-4B-Base, measuring how much of its RL-trained counterpart's gain each intervention recovers on 120 AIME problems.
- Power sampling with the same 8,192-token output limit brings no gain beyond greedy decoding.
- Yet borrowing the RL model's token only where the two models' greedy choices differ, roughly 4% of generated positions, recovers 90.5% of the pass@1 gain, with similar recovery across three model pairs on MATH-500.
- We find that even a single opening token has a substantial effect: forcing the base model to begin with "Alright" recovers 35% of the RL gain.
Sources (1)
- [1]What RL Brings to Reasoning: A Few Decisions the Base Model Would Not MakearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 12:34 PM
Reinforcement learning with verifiable rewards (RLVR) substantially improves mathematical reasoning, but what it adds to the base model remains unclear.
We investigate this through a sequence of generation-time interventions on Qwen3-4B-Base, measuring how much of its RL-trained counterpart's gain each intervention recovers on 120 AIME problems.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 17, 2026Nemotron 3.5 Bolt, Muse Spark Open, and a Fast LFM2.5 24B
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026ReLaDrift: Recurrent Language Generation via Drifting for Test-Time Scaling
- Oct 16, 2026Welcome Beam: a 501B open model you can own, customize, and scale
- Oct 15, 2026open source framework that makes your AI agents generate UI elements
- Oct 15, 2026Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint