Toward Online Robust Zero-Sum Markov Games with Function Approximation

Key points

• Existing robust reinforcement-learning theory does not yet provide such an operator-learning perspective beyond specialized multi-agent settings.1
• We develop an optimistic dual robust Bellman framework that combines dual robust Bellman fitting, optimistic minimax planning, backward-validated confidence sets, and exploiter-mix data collection.1
• Our analysis is organized around the robust Bellman residuals induced by the candidate value class and yields a regret reduction that separates exploration complexity, robust operator-estimation error, stage-game solution error, and misspecification.1
• Under explicit interpretable assumptions, we obtain conditional theorem interfaces for general and projected function approximation, together with closed results in several structured regimes, including tabular, epochwise cross-fit linear, self-normalized pure-online linear / finite-rank kernel, and projected RKHS truncation.1

Sources (1)

• [1]Toward Online Robust Zero-Sum Markov Games with Function Approximation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 12:38 PM

  Online robust zero-sum Markov games pose a coupled operator-learning and game-solving problem: from nominal interaction data, the learner must recover a worst-case minimax Bellman operator while maintaining strategic coverage and controlling stage-game approximation error.

  Existing robust reinforcement-learning theory does not yet provide such an operator-learning perspective beyond specialized multi-agent settings.

Extractive summary: sentences quoted from the sources.