Dissimilarity-Based Markov Decision Processes for Stabilizing State Distributions

Key points

• Reinforcement learning for stochastic systems typically optimizes expected cumulative rewards, while reliable and repeatable behavior also depends on how the state distribution evolves over time.1
• We formulate the problem as a Dissimilarity-based Markov Decision Process (DMDP), in which the policy acts on the current state distribution and the objective measures its dissimilarity to a target distribution.1
• We establish sufficient conditions under which the optimal policy makes the target distribution asymptotically stable.1
• These results quantify how learning and estimation errors affect the guarantees and identify conditions under which learned policies retain asymptotic stability.1

Sources (1)

• [1]Dissimilarity-Based Markov Decision Processes for Stabilizing State Distributions
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 12:56 AM

  To support policy learning, we introduce a finite-dimensional representation of distributions and show how stability guarantees for its parameter dynamics extend to the underlying state distributions.

  Reinforcement learning for stochastic systems typically optimizes expected cumulative rewards, while reliable and repeatable behavior also depends on how the state distribution evolves over time.

Extractive summary: sentences quoted from the sources.