SWIFT: Self-guided Weak-to-strong dIFfusion models via Target extrapolation
In this work, we introduce Swift, a simple yet effective self-guided training method that uses a frozen weak denoiser to improve a stronger model without adding inference overhead.
ProofPaper ↗
Key points
- Training high-quality diffusion models is computationally expensive, and inference-time guidance often adds further sampling cost.
- We explain this mechanism through a spectral analysis of finite-time population gradient flow in an RKHS.
- With an early-stopped weak guide, we prove that a nonempty range of extrapolation strengths yields strictly lower posterior-mean error than matched standard training, despite shifting the regression target away from the posterior mean.
- These results establish target extrapolation as an effective approach to weak-to-strong diffusion training with efficient single-model inference.
Sources (2)
- [1]SWIFT: Self-guided Weak-to-strong dIFfusion models via Target extrapolationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 05:57 AM
In this work, we introduce Swift, a simple yet effective self-guided training method that uses a frozen weak denoiser to improve a stronger model without adding inference overhead.
Training high-quality diffusion models is computationally expensive, and inference-time guidance often adds further sampling cost.
- [2]SWIFT: Self-guided Weak-to-strong dIFfusion models via Target extrapolationHugging Face Daily Papers · Oct 19, 12:00 AM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 18, 2026Steered Not Stirred: Effective Reward Tilting using Surrogate Reward Mixing
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026Logic-Guided Graph Denoising Diffusion Models
- Oct 16, 2026Data-Efficient Classifier Pretraining Through Diffusion Models
- Oct 16, 2026DIFFLET: A Fast Inference Engine for Diffusion Transformer Models on AWS Trainium
- Oct 16, 2026Learning to Be Guided: Shaping Latent Geometry for Classifier-Free Guidance
