Budgeted KV-Cache Repair for Evolving Contexts
We introduce budgeted KV-cache repair: recovering task performance by recomputing a user-chosen fraction of the retained suffix.
ProofPaper ↗
Key points
- Language-model agents revise their working context as information becomes obsolete, but deleting text does not erase its influence on cached computation.
- Later key-value (KV) states remain conditioned on the deleted content, while recomputing the entire retained suffix can be costly.
- Comparing repair allocations under identical relevance scores and token budgets, we find that refreshing coherent regions can substantially outperform refreshing isolated tokens.
- To execute these repairs efficiently, a layer-packed executor processes selected states together at each layer while preserving full causal cache access.
Sources (1)
- [1]Budgeted KV-Cache Repair for Evolving ContextsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 07:44 PM
We introduce budgeted KV-cache repair: recovering task performance by recomputing a user-chosen fraction of the retained suffix.
Language-model agents revise their working context as information becomes obsolete, but deleting text does not erase its influence on cached computation.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026Alibaba Qwen Introduces QwenGyre: Elastic Reinforcement Learning RL for XLong-Horizon Agents
- Oct 12, 2026AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers
- Oct 11, 2026Same Model, Different Speed: Why Your Inference Provider Matters — FriendliAI
- Oct 11, 2026autotrust/JEV-35B-NVFP4
- Oct 11, 2026History Replays Itself: Unleashing Long-Context Capability in Linear Attention
- Oct 10, 2026vllm-sr/d3