Dancing Diffusion: Learning Continuous Programs for Image Generation and Editing

Key points

• Diffusion models generate samples through a multi-step denoising trajectory, yet condition every step on the same instruction, so the trajectory cannot be addressed at the level of individual operations.1
• The compiler is trained through the images it produces, with visual feedback on both the requested change and the retained content.1
• On SANA, reusing an executed prefix preserves markedly more of the scene than regenerating from noise at a similar strength of edit, learned conditions preserve more background than native prompt switching from the same saved state, and the learned program improves generation quality over native SANA.1
• By converting a single-instruction trajectory into a program of addressable operations, Dancing Diffusion lays a foundation for structured intermediate computation in diffusion models.1

Sources (1)

• [1]Dancing Diffusion: Learning Continuous Programs for Image Generation and Editing
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 10:46 AM

  We introduce Dancing Diffusion, which assigns each span of timesteps a dedicated continuous condition block and thereby turns conditioning into a latent program.

  Diffusion models generate samples through a multi-step denoising trajectory, yet condition every step on the same instruction, so the trajectory cannot be addressed at the level of individual operations.

Extractive summary: sentences quoted from the sources.