Certifying Almost-Sure Reachability in Unknown MDPs
{\em Specification-Guided Reinforcement Learning (RL)} has largely concerned itself with learning an optimal policy against a formal specification in an unknown {\em Markov Decision Process (MDP)}.
ProofPaper ↗
Key points
- Given a policy already in hand, can one verify that it is correct against a formal specification in an unknown MDP, using sampling access alone?
- We investigate it for the {\em almost-sure reachability property}, whose correctness is witnessed by a {\em ranking supermartingale (RSM)} certificate.
- On the positive side, we show that, under a mild restriction, a known lower bound $p{\min}$ on nonzero transition probabilities, a correct certificate can be learned with PAC-style guarantees.
- Finally, our experiments show that the $p{\min}$-based sufficient condition, while sound, is conservative: the learned certificate often remains valid on the true MDP even when the condition fails.
Sources (1)
- [1]Certifying Almost-Sure Reachability in Unknown MDPsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 10:33 AM
{\em Specification-Guided Reinforcement Learning (RL)} has largely concerned itself with learning an optimal policy against a formal specification in an unknown {\em Markov Decision Process (MDP)}.
Given a policy already in hand, can one verify that it is correct against a formal specification in an unknown MDP, using sampling access alone?
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026Fish Audio Drama 3 Technical Report
- Oct 13, 2026Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes
- Oct 13, 2026GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning
- Oct 12, 2026Z.ai GLM-Legal Beats Claude Fable 5 and Kimi K3 on Harvey LAB
- Oct 12, 2026Alibaba Qwen Introduces QwenGyre: Elastic Reinforcement Learning RL for XLong-Horizon Agents
- Oct 11, 2026AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards