Variance-Aware Non-Stationary Bandits:All-Budget Regret Bounds and Adaptation Costs

Key points

• Low reward variance reduces estimation cost, but does not eliminate monitoring of arms that may improve while unobserved.1
• Let $A$ be the number of arms, $T$ the horizon, $B$ a bound on consecutive mean variation excluding the initial vector, and $Q$ a bound on the sum of the largest per-round reward variances.1
• For every admissible pair $Q,B$, the minimax expected dynamic regret is, within a single logarithmic factor, \[1
• We also quantify adaptation costs: on a disclosed deterministic one-improvement family, a stationary-regret budget has a matching worst-case change-regret frontier, including a finite improvement pulse.1

Sources (1)

• [1]Variance-Aware Non-Stationary Bandits:All-Budget Regret Bounds and Adaptation Costs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 10:10 AM

  We refine the variance--variation tradeoff and characterize how both costs depend on supplied variance and variation budgets in bounded, non-stationary multi-armed bandits.

  Low reward variance reduces estimation cost, but does not eliminate monitoring of arms that may improve while unobserved.

Extractive summary: sentences quoted from the sources.