Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control

Key points

• We introduce Transport REINFORCE, a transport map-based approach that perturbs a suitable transformation of the population distribution to estimate this missing mean-field contribution.1
• In continuous state spaces, we project the population distribution onto the manifold of Gaussian mixtures, and then randomize it via a transport map that ensures the perturbed law remains within this manifold.1
• We prove consistency of the perturbed objective and gradient as the perturbation vanishes, and derive bias and mean-square error bounds for the resulting sample-based gradient estimator.1
• Numerical experiments on several MFC benchmarks show that Transport REINFORCE improves over standard REINFORCE.1

Sources (1)

• [1]Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 09:58 AM

  We develop a model-free policy gradient method for discrete-time mean-field control (MFC).

  We introduce Transport REINFORCE, a transport map-based approach that perturbs a suitable transformation of the population distribution to estimate this missing mean-field contribution.

Extractive summary: sentences quoted from the sources.