SpecKCast: Spectral Kalman Forecasting for Diffusion Acceleration

Key points

• Feature caching reduces this cost by reusing features across denoising steps.1
• We propose SpecKCast, a training-free method that forecasts the model output in the frequency domain.1
• SpecKCast applies a 2D discrete cosine transform (DCT) to the output and tracks each coefficient with a lightweight Kalman filter along a band-specific time coordinate derived from the noise schedule.1
• Experiments on FLUX.1-dev, Qwen-Image, Wan2.1-14B, and HunyuanVideo show that SpecKCast achieves a better trade-off between acceleration and fidelity than existing caching and forecasting methods, reaching up to 6x speedup with less than 0.05 GiB of additional memory.1

Sources (2)

• [1]SpecKCast: Spectral Kalman Forecasting for Diffusion Acceleration
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 05:58 PM

  Diffusion Transformers (DiTs) achieve high-fidelity image and video generation, but they suffer from substantial inference costs due to repeated transformer evaluations across many denoising steps.

  Feature caching reduces this cost by reusing features across denoising steps.

• [2]SpecKCast: Spectral Kalman Forecasting for Diffusion Acceleration
  Hugging Face Daily Papers · Oct 19, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.