SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents

Key points

• Reinforcement learning (RL) for search agents often suffers from evidence fabrication, a failure mode where agents bypass external information ($E{ext}$) and fabricate an incorrect answer.1
• We argue that this fabrication arises from an over-reliance on unreliable internal knowledge ($E{int}$) as shortcuts due to the inherent opacity of reasoning trajectories.1
• SOAR (i) disentangles $E{ext}$ and $E{int}$ into parallel, observable streams via a Source-Aware Workflow, and (ii) incorporates Source-Consistent Reward to guide the policy evolution.1
• By making information-source attribution explicit, SOAR enables precise credit assignment between different knowledge streams.1

Sources (1)

• [1]SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 07:45 AM

  To address this, we propose SOAR (SOurce-Aware Reinforcement learning), a paradigm that transforms monolithic reasoning into source-explicit trajectories.

  Reinforcement learning (RL) for search agents often suffers from evidence fabrication, a failure mode where agents bypass external information ($E_{ext}$) and fabricate an incorrect answer.

Extractive summary: sentences quoted from the sources.