Cache Locally, Recompute Selectively for Streaming Video Understanding
We propose FrameBlend, a simple yet effective plug-and-play method to address this mismatch through selective KV recomputation, without requiring any additional training.
ProofPaper ↗
Key points
- Streaming video understanding (SVU) requires video-LLMs to process incoming video continuously while answering user queries with low latency.
- Recent methods reduce query-time cost by prefilling frames into KV caches during idle time and reusing the cached KVs at query time.
- To bound per-frame computation over unbounded video streams, these methods restrict the context available during prefill.
- At query time, FrameBlend re-prefills selected video tokens using query-relevant context and replaces their cached KVs with recomputed ones.
Sources (1)
- [1]Cache Locally, Recompute Selectively for Streaming Video UnderstandingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 05:39 AM
We propose FrameBlend, a simple yet effective plug-and-play method to address this mismatch through selective KV recomputation, without requiring any additional training.
Streaming video understanding (SVU) requires video-LLMs to process incoming video continuously while answering user queries with low latency.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch
- Oct 20, 2026Gradium Open-Sources Phonon, a 31M-Parameter On-Device TTS Model Under Apache 2.0
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 20, 2026MPS in PyTorch 2.15
- Oct 19, 2026Introducing Clairic, a new System One class of extraction models
- Oct 19, 2026NVIDIA Cosmos3-H-Dreams: Bringing Clinical Surgery to Life Through Generative Simulation