PrT-Flow: Probability-Tractable Flow Matching for Reinforcement Learning

Key points

• Diffusion models and Flow Matching can represent flexible distributions over robot actions, but many reinforcement learning algorithms also require the probability of each action.1
• We train these finite steps with Conditional Euler-step Training (CET), which supervises their displacement over one step during sampling rather than their instantaneous velocity.1
• PrT-Flow achieves 96% success on Robomimic Lift after 300 epochs of imitation learning, matching the diffusion policy reference, and 96% success on OGBench Cube-Triple after 600k online steps.1
• These results demonstrate that invertible networks with tractable probability densities can learn effective robot policies and support SAC-style actor-critic training.1

Sources (1)

• [1]PrT-Flow: Probability-Tractable Flow Matching for Reinforcement Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 03:51 AM

  We introduce PrT-Flow (Probability-Tractable Flow Matching), which computes probability density while retaining multimodal, multi-step action generation.

  Diffusion models and Flow Matching can represent flexible distributions over robot actions, but many reinforcement learning algorithms also require the probability of each action.

Extractive summary: sentences quoted from the sources.