Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling

Key points

• LLM agents are commonly trained with RLVR, but the sparse nature of typical RLVR rewards makes credit assignment challenging, especially for long-horizon tasks.1
• Self-Credit builds on group-based RL with three components.1
• On BrowseComp-Plus, Self-Credit improves over the base models by 37.0 and 36.2 points on Qwen3 8B and 32B respectively, and outperforms the strongest agentic RL baselines by 8.5 and 5.6 points, respectively.1
• Analyses show that self-generated credit provides useful signals for selecting critical actions, and that intra-group rescaling, especially on failed rollouts, is important across model scales.1

Sources (2)

• [1]Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling
  Hugging Face Daily Papers · Oct 20, 12:00 AM

  We propose Self-Credit, an end-to-end agentic RL method that creates dense supervision signals from sparse outcome rewards using self-generated credit assignment via intra-group rescaling.

  LLM agents are commonly trained with RLVR, but the sparse nature of typical RLVR rewards makes credit assignment challenging, especially for long-horizon tasks.

• [2]Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:38 PM · same content

Extractive summary: sentences quoted from the sources.