AdvFidelity: Unifying Preference Alignment and Few-Step Image Generation within Reinforcement Learning

Key points

• To address this gap, we propose AdvFidelity, which integrates artifact-sensitive fidelity supervision into RL through an online adversarial reward.1
• Reconstructing moderately noised references largely preserves style and scene layout while exposing low-level degradations.1
• Built on LeapAlign, AdvFidelity backpropagates gradients from both rewards through the few-step sampling trajectories used at inference.1
• Across two model backbones and three task settings, AdvFidelity matches full-step LeapAlign in alignment, task performance, and overall generation quality and remains highly competitive with other evaluated full-step RL methods, using only 4 or 8 steps without CFG.1

Sources (1)

• [1]AdvFidelity: Unifying Preference Alignment and Few-Step Image Generation within Reinforcement Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 08:09 AM

  Practical image generation requires both human preference alignment and few-step efficiency, yet combining reinforcement learning (RL) with distillation is often multi-stage and may erode RL gains We observe that few-step RL progressively improves structural coherence, even with specialized rewards, but leaves low-level artifacts that these rewards may fail to penalize.

  To address this gap, we propose AdvFidelity, which integrates artifact-sensitive fidelity supervision into RL through an online adversarial reward.

Extractive summary: sentences quoted from the sources.