FlashControl: One-Step Controllable Generator via Distillation-Friendly Single-Stream Teachers

Key points

• While recent distillation methods enable one-step text-to-image generation, extending these techniques to controllable generation is still challenging.1
• Instead of adding extra conditional branches, we finetune a selected subset of layers in the original text-to-image diffusion model to directly enable controllable generation.1
• This results in DeltaControl, a single-flow, multi-step diffusion model capable of supporting spatial-condition generation while matching the performance of dual-branch approaches.1
• Building on this teacher model, we further apply step-distillation to obtain FlashControl, a one-step model capable of spatial controllable generation with a single forward pass.1

Sources (1)

• [1]FlashControl: One-Step Controllable Generator via Distillation-Friendly Single-Stream Teachers
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 06:32 PM

  Diffusion models have become a leading approach in visual generative modeling, particularly effective in text-to-image and controllable generation.

  While recent distillation methods enable one-step text-to-image generation, extending these techniques to controllable generation is still challenging.

Extractive summary: sentences quoted from the sources.