TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation

Key points

• Existing KV cache quantization methods are primarily designed for LLMs and transfer poorly to video models, where activations exhibit different statistics and strong spatiotemporal redundancy.1
• We observe that the video KV cache is largely predictable over time, with a clear asymmetry.1
• Based on these observations, we propose \method, a training-free predictive KV cache quantization framework for AR video generation. \method predicts keys and values according to their distinct dependencies and stores only the residuals at low precision.1
• Experiments on Self-Forcing, Causal Forcing, and LongCat-Video-13B show that \method consistently achieves higher generation quality than existing KV cache quantization methods at comparable or higher compression ratios, reaching up to $6.71\times$ KV cache compression with the best overall quality--efficiency tradeoff.1

Sources (1)

• [1]TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 03:59 AM

  Autoregressive (AR) video diffusion models enable streaming generation of long videos but suffer from significant memory overhead due to the rapidly growing KV cache.

  Existing KV cache quantization methods are primarily designed for LLMs and transfer poorly to video models, where activations exhibit different statistics and strong spatiotemporal redundancy.

Extractive summary: sentences quoted from the sources.