Cascading Bandits with Noisy Observations: Algorithms and Regret Bounds

Key points

• For the case where both selection and ordering scores exist, we provide a UCB-based algorithm with $O((L-K)\log T)$ instance-dependent regret over a time horizon $T$, which matches an algorithm-agnostic lower bound (up to constant factors).1
• For the more challenging scenario where scores don't exist, we develop a new algorithmic template, DP-UCB, that embeds optimism directly inside an exact dynamic program, yielding $O\big(LK\log T\big)$ regret.1
• Finally, we validate our theoretical results through extensive numerical evaluations and simulations over both synthetic and real-world datasets.1

Sources (1)

• [1]Cascading Bandits with Noisy Observations: Algorithms and Regret Bounds
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 08:10 AM

  We propose a novel stochastic cascading bandit setup where probing an ordered shortlist of $K$ out of $L$ arms incurs position-dependent costs, and the learner observes only noisy feedback regarding each arm's true output.

  For the case where both selection and ordering scores exist, we provide a UCB-based algorithm with $O((L-K)\log T)$ instance-dependent regret over a time horizon $T$, which matches an algorithm-agnostic lower bound (up to constant factors).

Extractive summary: sentences quoted from the sources.