PixARG: Pixel-Asynchronous Retrieval-Augmented Diffusion for Fast and Realistic Image Generation

Key points

• While diffusion models achieve impressive image generation capabilities, they suffer from high computational costs, implausible artifacts, and limited adaptability to dynamic or private data due to static, centralized training.1
• Specifically, PixARG constructs a pixel-level affinity map between the user's text prompt and the retrieved image to guide a heterogeneous noise initialization.1
• To accommodate this spatially non-uniform noise and ensure synchronized generation, we introduce a pixel-wise asynchronous denoising scheduler and a spatiotemporal denoiser conditioned on the affinity map.1
• Furthermore, we derive an affinity-adaptive training objective that elegantly balances deterministic restoration with stochastic regeneration across pixels.1

Sources (1)

• [1]PixARG: Pixel-Asynchronous Retrieval-Augmented Diffusion for Fast and Realistic Image Generation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 08:29 AM

  To address these, we propose PixARG, a novel retrieval-augmented generative framework that integrates dynamic real-world image retrieval into the diffusion pipeline to simultaneously enhance efficiency and visual realism.

  While diffusion models achieve impressive image generation capabilities, they suffer from high computational costs, implausible artifacts, and limited adaptability to dynamic or private data due to static, centralized training.

Extractive summary: sentences quoted from the sources.