What Should We Remove? Evaluating the Reliability and Costs of RLVR Data Decontamination
Reinforcement learning with verifiable rewards (RLVR) math corpora are routinely screened for benchmark overlap, but the reliability and data-loss costs of cleaning policies remain underexamined.
ProofPaper ↗
Key points
- We audit sixteen open RLVR math corpora comprising 1,673,514 corpus-specific training items against six benchmarks containing 1,949 questions.
- We evaluate eight policies based on exact, lexical, and semantic matching using 594 independently and blindly dual-reviewed training-benchmark pairs.
- A separate review of all 83 candidate matches for 80 sampled training items measures item-level false removal.
- We release reviewed labels, per-policy decisions, and coverage accounting to support evaluation of cleaning reliability and cost rather than overlap counts alone.
Sources (1)
- [1]What Should We Remove? Evaluating the Reliability and Costs of RLVR Data DecontaminationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 04:19 AM
Reinforcement learning with verifiable rewards (RLVR) math corpora are routinely screened for benchmark overlap, but the reliability and data-loss costs of cleaning policies remain underexamined.
We audit sixteen open RLVR math corpora comprising 1,673,514 corpus-specific training items against six benchmarks containing 1,949 questions.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards
- Oct 14, 2026mistralai/Timestral-209M
- Oct 14, 2026Cortex: Consolidating Context into Parametric Memory
- Oct 14, 2026ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models
- Oct 14, 2026Player-1: Learning to Play from Synthetic Experience
- Oct 13, 2026Gemini 4 Argon for Robotics