Best Arm Identification for Bandits with Shifting Means

Key points

• While classically the mean rewards of the $K$ arms are stable in time, in Shifting Means only the gaps $\boldsymbolΔ$ between mean rewards are stable, while their common shift may be determined adversarially in each round.1
• Handling shifts requires new tools: we show that algorithms employing a Generalized Likelihood Ratio Test (GLRT) stopping rule, including the popular Track-and-Stop, fail under time-varying shifts.1
• Instead, we propose Importance Weights for Shifting Means ($\mathsf{ISM}$).1
• Assuming means bounded by $U$ and $σ^2$-sub-Gaussian rewards, we show $\mathsf{ISM}$ to be $δ$-correct and to enjoy a sample complexity bound of order $K (σ^2 + U^2) Δ{\min}^{-2} \ln \frac{1}δ$.1

Sources (1)

• [1]Best Arm Identification for Bandits with Shifting Means
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 05:44 PM

  We study the best arm identification problem in a stochastic environment with a novel form of adversarial perturbations, which we coin Shifting Means.

  While classically the mean rewards of the $K$ arms are stable in time, in Shifting Means only the gaps $\boldsymbolΔ$ between mean rewards are stable, while their common shift may be determined adversarially in each round.

Extractive summary: sentences quoted from the sources.