Memorization and Malign Generalization in Conditional Diffusion Models with Random Features
Conditional diffusion models generate diverse, novel, and high-quality samples under prescribed conditions.
ProofPaper ↗
Key points
- In this work, we analyze a random-feature conditional score model in the high-dimensional proportional limit, deriving asymptotic expressions for training and test losses.
- By decomposing the test loss, we show that in the overparameterized regime, increasing model width improves prediction of the condition-dependent mean while reducing within-condition prediction variance, a phenomenon we term "malign generalization.
- Furthermore, analyzing the training loss reveals that more informative conditions lead to memorization of training samples at smaller widths.
- These theoretical findings are supported by experiments with U-Net architectures on realistic data.
Sources (1)
- [1]Memorization and Malign Generalization in Conditional Diffusion Models with Random FeaturesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:52 AM
Conditional diffusion models generate diverse, novel, and high-quality samples under prescribed conditions.
In this work, we analyze a random-feature conditional score model in the high-dimensional proportional limit, deriving asymptotic expressions for training and test losses.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 8, 2026Diffusion Meta-Prompting and Steering for Generalizable Foundation Model Adaptation
- Oct 8, 2026LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation
- Oct 8, 2026DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models
- Oct 8, 2026The Lattice of Transition Laws
- Oct 7, 2026Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning
- Oct 7, 2026SALD: Self-Referenced Advantage Learning for Diffusion Models