RH-Detect: A Unified Benchmark for Reward Hacking Detection
We present RH-Detect, a benchmark that combines reward-hacking-relevant subsets from eleven public datasets, comprising 92,761 rows and six behavior categories, into a common schema.
ProofPaper ↗
Key points
- Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model systems.
- On 5,021 open-ended evaluation units, each comprising a task prompt and a free-form model continuation, including multi-turn tool-use trajectories, we evaluate six off-the-shelf language models from five families as reward hacking detectors without additional training.
- We find that different input formats have different effects across models.
- Our results show that a single pooled score can conceal variation across data sources, detector inputs, and training procedures.
Sources (1)
- [1]RH-Detect: A Unified Benchmark for Reward Hacking DetectionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 09:52 PM
We present RH-Detect, a benchmark that combines reward-hacking-relevant subsets from eleven public datasets, comprising 92,761 rows and six behavior categories, into a common schema.
Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model systems.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 7, 2026SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages
- Oct 7, 2026On the Clock: Towards Punctual and Productive Time-Budgeted AI Agents
- Oct 7, 2026RECAST: Learning to Compute the Right Context through Adaptive Evidence Routing
- Oct 6, 2026Quantization Effects on Tool-Failure Recovery Vary Across Prompts and Evaluation Designs
- Oct 6, 2026FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents
- Oct 6, 2026EMHO: EMbodied Agent Harness Optimization via Experience Traces