Training-Free Prompt Selection for Efficient Reinforcement Learning With Partial Rollouts

Key points

• Since training examples contribute unequally to learning, selecting a small, informative subset can offer a more efficient alternative to using the entire data pool.1
• The training-free selection method estimates prompt value with hidden states from complete greedy rollouts.1
• We introduce LIFT, a training-free data selector that identifies impactful prompts using only partial reasoning rollouts.1
• At its core, LIFT uses Partial Rollout Context Lift (PRCL), a score inspired by pointwise $\mathcal{V}$-information that accumulates token-level log-likelihood ratios between a question-conditioned context and a null context with the question removed.1

Sources (1)

• [1]Training-Free Prompt Selection for Efficient Reinforcement Learning With Partial Rollouts
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 08:50 PM

  Reinforcement learning with verifiable rewards (RLVR) has emerged as a prominent paradigm for training reasoning models, but its computational cost makes efficient data selection increasingly important.

  Since training examples contribute unequally to learning, selecting a small, informative subset can offer a more efficient alternative to using the entire data pool.

Extractive summary: sentences quoted from the sources.