EG-GRPO: Evidence-Guided Step-level Credit Assignment for Deep Search Agents
In this paper, we propose Evidence-Guided Group Relative Policy Optimization (EG-GRPO), a framework that derives step-level supervision from the evidence annotations available in synthetic deep search data.
ProofPaper ↗
Key points
- LLM-based search agents demonstrate strong capabilities in long-horizon information seeking and reasoning, typically trained via reinforcement learning.
- By representing the accumulated evidence as an evolving knowledge state, EG-GRPO further performs state-conditioned relative advantage estimation and combines the resulting step-level signal with the standard trajectory-level outcome advantage.
- Experiments on five challenging deep search benchmarks show that EG-GRPO generally outperforms standard GRPO and existing reward-enhanced and step-level RL baselines across different model scales.
- These results demonstrate that task-grounded evidence annotations can provide effective process supervision for credit assignment in deep search RL.
Sources (1)
- [1]EG-GRPO: Evidence-Guided Step-level Credit Assignment for Deep Search AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 01:39 PM
In this paper, we propose Evidence-Guided Group Relative Policy Optimization (EG-GRPO), a framework that derives step-level supervision from the evidence annotations available in synthetic deep search data.
LLM-based search agents demonstrate strong capabilities in long-horizon information seeking and reasoning, typically trained via reinforcement learning.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 18, 2026Indistinguishable Agents: Training Agent Privacy as Twin-World Indistinguishability
- Oct 18, 2026Trackable Skill Library Lifecycle Management for Agentic Reinforcement Learning
- Oct 17, 2026ProbePO: Sparse Probes, Redistributed Credit for Critic-Free LLM Reinforcement Learning
- Oct 17, 2026HybridResearcher: Bridging the Structured-Unstructured Boundary for Deep Research Agents
- Oct 15, 2026AlphaBet: Reinforcement Learning for LLM Event Forecasting Agents
- Oct 15, 2026Learning to Steer LLM Agents and Multi-Agent Systems