Minimax estimation of the pseudo-spectral gap of continuous Markov chains, with applications to empirical concentration bounds

Key points

• Extending earlier works on finite-state space chains by Wolfer and Kontorovich (2024), we provide an estimator in the case of a compact, continuous state-space.1
• Our estimator appears to be minimax-optimal on the class of chains with a transition operator with Lipschitz density, up to logarithmic terms.1
• Our estimator, combined with Paulin's version of Bernstein inequality, allows to derive empirical concentration inequalities for Markov chains.1
• In particular, for rested bandits with Markov rewards, we provide a version of UCB with a regret bound similar to the one in the iid case, without assuming any knowledge on the pseudo-spectral gap of the chains.1

Sources (1)

• [1]Minimax estimation of the pseudo-spectral gap of continuous Markov chains, with applications to empirical concentration bounds
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 11:22 AM

  We study the minimax estimation of the pseudo-spectral gap $γ_{ps}$ of a Markov chain from the observation of a single trajectory.

  Extending earlier works on finite-state space chains by Wolfer and Kontorovich (2024), we provide an estimator in the case of a compact, continuous state-space.

Extractive summary: sentences quoted from the sources.