IR-PRM+ with Previous-Utility Prediction: Constant Regret and Best-Iterate Convergence

Key points

• Regret matching (and its variants) and counterfactual regret minimization are central algorithms in practice for normal-form and extensive-form game solving.1
• For simultaneous and alternating IR-PRM+ in finite two-player zero-sum matrix games, previous-utility prediction recovers the constant-regret and convergence-rate guarantees previously obtained with extra-gradient prediction: external regret bounded independently of T, an O(T^{-1}) average duality gap, and an O(T^{-1/2}) best-current-iterate gap.1
• The key step bounds the component of consecutive strategy movement not controlled by the existing IR-PRM+ regret inequality using the relative growth of the nondecreasing regret-vector norm rhot.1
• We extend the analysis to finitely many simplex learners under a weighted Minty condition and, through the corresponding CFR variant IR-PCFR+, recover analogous guarantees for finite two-player zero-sum perfect-recall extensive-form games under the same full-reach condition as prior work.1

Sources (1)

• [1]IR-PRM+ with Previous-Utility Prediction: Constant Regret and Best-Iterate Convergence
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 06:10 PM

  Recent work introduced IREG-PRM+, a regret matching variant which does have strong theoretical convergence guarantees.

  Regret matching (and its variants) and counterfactual regret minimization are central algorithms in practice for normal-form and extensive-form game solving.

Extractive summary: sentences quoted from the sources.