Routing Policy Optimization for Online Diffusion Reinforcement Learning
Reinforcement learning can align visual generators with reward signals beyond their pretraining objectives, but existing score-function methods typically introduce stochasticity by perturbing the reverse denoising trajectory, leading to costly rollout-based optimization.
ProofPaper ↗
Key points
- We instead ask whether stochastic exploration can be introduced inside the generator while preserving its original sampler.
- Routing Policy Optimization (RPO) answers this question by turning attention routing into a stochastic exploration action.
- At inference, RPO restores dense attention and retains the original deterministic sampler.
- Experiments on text-to-image and text-to-video generation show that RPO improves generation quality while achieving nearly 4x higher training efficiency than Flow-GRPO-style methods.
Sources (2)
- [1]Routing Policy Optimization for Online Diffusion Reinforcement LearningHugging Face Daily Papers · Oct 15, 12:00 AM
Reinforcement learning can align visual generators with reward signals beyond their pretraining objectives, but existing score-function methods typically introduce stochasticity by perturbing the reverse denoising trajectory, leading to costly rollout-based optimization.
We instead ask whether stochastic exploration can be introduced inside the generator while preserving its original sampler.
- [2]Routing Policy Optimization for Online Diffusion Reinforcement LearningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 05:40 PM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO
- Oct 14, 2026Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning
- Oct 14, 2026The Design Space of Tri-Modal Masked Diffusion Models
- Oct 13, 2026PairIT: Autoregressive Transformers for Low-Data Molecule Optimization
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
- Oct 13, 2026APT: Feedback-Driven Agentic Post-Training for Text-to-Image Generation