Learning to Program Retrievals: Composing Search Primitives via Reinforcement Learning

Key points

• Retrieval-augmented language models increasingly use multi-step search to solve knowledge-intensive tasks, yet existing systems typically expose retrieval through fixed pipelines or collections of atomic tools.1
• To address this challenge, we propose program mutation, a grammar-constrained and execution-verified procedure that generates local variants of suboptimal programs sampled during RL, converts them into training trajectories, and distills them back into the policy.1
• We evaluate RPL on open-domain question answering and corpus-level reasoning tasks, comparing it with a range of retrieval and reasoning baselines.1
• Our results show that programmable search expands the retrieval strategies that agents can learn, while program mutation provides an effective mechanism for exploring useful program structures beyond direct on-policy sampling.1

Sources (1)

• [1]Learning to Program Retrievals: Composing Search Primitives via Reinforcement Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 08:03 PM

  We introduce Retrieval Program Learning (RPL), a programmable search framework that allows a policy to compose search primitives into executable programs with local state and control flow.

  Retrieval-augmented language models increasingly use multi-step search to solve knowledge-intensive tasks, yet existing systems typically expose retrieval through fixed pipelines or collections of atomic tools.

Extractive summary: sentences quoted from the sources.