DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement Learning
We introduce DEFlow, a deep equilibrium flow policy that replaces explicit Euler sampling with implicit Euler integration and solves all intermediate actions jointly as an equilibrium system.
ProofPaper ↗
Key points
- Expressive flow policies can represent multiple high-value actions, but sampling them with few integration steps while retaining tractable action densities remains challenging.
- We combine implicit differentiation with a discrete change-of-variables formula for the final-action density, enabling off-policy training within soft actor-critic without backpropagating through solver iterations.
- DEFlow achieves an average relative improvement of approximately 10% in mean successful coverage over the strongest baseline across four Multi-Goal docking variants.
- These results support implicit flow policies for tasks requiring both diversity and precision, at the cost of additional nonlinear and linear solves.
Sources (1)
- [1]DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement LearningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 11:18 AM
We introduce DEFlow, a deep equilibrium flow policy that replaces explicit Euler sampling with implicit Euler integration and solves all intermediate actions jointly as an equilibrium system.
Expressive flow policies can represent multiple high-value actions, but sampling them with few integration steps while retaining tractable action densities remains challenging.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026X-Ragent: Tool-Augmented Reinforcement Learning for Evidence-Grounded Chest X-Ray Reasoning
- Oct 12, 2026StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions
- Oct 12, 2026LongR: Unleashing Long-Context Reasoning via Reinforcement Learning with Dense Utility Rewards
- Oct 11, 2026AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards
- Oct 11, 2026RSIPO: Recursive Self-Improving Policy Optimization for Agentic Reinforcement Learning via Skill-Policy Co-Evolution
- Oct 11, 2026ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs