Continuous-Time Policy Gradient from Discrete Data: Convergence Analysis and Model-Free Algorithms
Continuous-time reinforcement learning seeks an optimal policy under unknown dynamics driven by stochastic differential equations using observations collected at discrete times.
ProofPaper ↗
Key points
- We develop a policy gradient framework that combines rigorous convergence analysis with learning from discrete data.
- We first establish a performance difference lemma and derive policy gradient formulas for randomized and parametrized policies.
- The resulting on-policy and off-policy model-free actor-critic algorithms preserve the continuous-time structure of the control problem.
- Numerical experiments on an entropy-regularized linear-quadratic regulator and a nonlinear wheeled robot demonstrate the performance and stability of the proposed methods with both known and data-driven dynamics.
Sources (1)
- [1]Continuous-Time Policy Gradient from Discrete Data: Convergence Analysis and Model-Free AlgorithmsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:33 PM
Continuous-time reinforcement learning seeks an optimal policy under unknown dynamics driven by stochastic differential equations using observations collected at discrete times.
We develop a policy gradient framework that combines rigorous convergence analysis with learning from discrete data.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model
- Oct 19, 2026Sparse RL Relaxation for LLM Reasoning
- Oct 19, 2026Training Natural Schemers: RL Can Amplify Oversight-Conditional Behavior
- Oct 19, 2026Long-Horizon Multi-Object Rearrangement via Distilled Tree Search over Continuous Actions
- Oct 19, 2026TorchNeuron: running HF models on AWS Trainium is easier than ever
- Oct 19, 2026HarnessRL: Co-Evolving Policies with a Just-in-Time Training Harness for Open-Ended RL