PEARL: Process-Experience Aware Reinforcement Learning for Multi-Turn Agents
We introduce PEARL (Process-Experience Aware Reinforcement Learning), which improves local credit assignment without an additional critic.
ProofPaper ↗
Key points
- Terminal rewards make multi-turn agents trainable without step-level annotations, but group-relative advantages provide little guidance about which intermediate decisions to adjust.
- Given two continuations from the same history with different outcomes, PEARL locates their first divergence and builds a feature representation of each branch's local decision.
- Using this pattern memory, PEARL selectively strengthens negative advantages at lower-reward decisions, while leaving positive advantages and the base policy optimizer unchanged.
- Ablations of shaping location, success-bank filtering, and memory window support the value of selective, recurring-pattern-based updates.
Sources (1)
- [1]PEARL: Process-Experience Aware Reinforcement Learning for Multi-Turn AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 06:35 AM
We introduce PEARL (Process-Experience Aware Reinforcement Learning), which improves local credit assignment without an additional critic.
Terminal rewards make multi-turn agents trainable without step-level annotations, but group-relative advantages provide little guidance about which intermediate decisions to adjust.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 10, 2026Vela: User Behavior as Supervision Sparks LLM Intelligence
- Oct 9, 2026Visualizing the World Model in an Agent's Value Function
- Oct 9, 2026The Thinking Leakage in Reinforcement Learning for LLMs
- Oct 9, 2026Branching On-Policy Distillation
- Oct 9, 2026Rethinking Relative Policy Differences Transfer of On-Policy Distillation
- Oct 9, 2026SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models
