Reinforcement Learning with Conformal Action Sets: An Application to Sequential Recommendation

Key points

• Sequential recommenders typically use a fixed slate size even though the number of useful alternatives changes within a session.1
• We prove a deterministic bound on the observed proxy miss rate along adaptive trajectories.1
• To quantify the effect of pruning on reward, we derive an exact decomposition of value loss into filtering and selection losses.1
• Under explicit proxy and critic approximation conditions, this decomposition yields a finite session reward bound that also accounts for imperfect selection and set truncation, without requiring the learning parameters to converge.1

Sources (1)

• [1]Reinforcement Learning with Conformal Action Sets: An Application to Sequential Recommendation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 05:40 PM

  We propose Reinforcement Learning with Calibrated Pruning (RLCP), which adapts the retained action set using critic scores and an online threshold.

  Sequential recommenders typically use a fixed slate size even though the number of useful alternatives changes within a session.

Extractive summary: sentences quoted from the sources.