DualHeads: Reward-tilted few-step diffusion made simple
Distilled diffusion models generate high-quality images in a few sampling steps, but aligning them with task requirements and human preferences is difficult.
ProofPaper ↗
Key points
- Methods developed for multi-step diffusion models rely on differentiable rewards, or on score fields and stochastic trajectories that distilled generators may lack.
- We ask whether a few-step generator can be adapted using only external reward evaluations and reference samples, and show that two binary classifiers suffice.
- Our method, DualHeads, implements them as lightweight heads on a frozen diffusion backbone.
- Empirically, with both rule-based and model-based rewards, a few-step DualHeads generator reaches task rewards close to those of 40-step reinforcement learning methods while preserving image quality, at no extra inference cost.
Sources (1)
- [1]DualHeads: Reward-tilted few-step diffusion made simplearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 07:31 PM
Distilled diffusion models generate high-quality images in a few sampling steps, but aligning them with task requirements and human preferences is difficult.
Methods developed for multi-step diffusion models rely on differentiable rewards, or on score fields and stochastic trajectories that distilled generators may lack.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026When Do Gaussian Priors Fail? A Latent Geometry Investigation of RL Fine-Tuning
- Oct 20, 2026Large Flow Language Models
- Oct 20, 2026Matrix-Game 4.0: Harnessing Video Models for Real-Time Playable Game Worlds
- Oct 19, 2026NVIDIA Cosmos3-H-Dreams: Bringing Clinical Surgery to Life Through Generative Simulation
- Oct 19, 2026TamedFlow: Trajectory-Aware Distillation\\for Teacher-Assisted Diffusion Sampling
- Oct 19, 2026Functional Cross-layer Experts Routing for One-step Diffusion Distillation