Cache Engineering for Agents
Large Language Model (LLM) agents can handle increasingly complex tasks, navigating long trajectories of observations and actions.
ProofPaper ↗
Key points
- Prior work uses context engineering to reduce cache size by trimming or summarizing history.
- We propose cache engineering as a new scheme.
- We formalize these cache operations and identify history dependence as a serving challenge: identical token sequences can yield different cache states under different cache operation sequences, making prefix matching insufficient for correct cache reuse.
- Experiments show that a tool-response rolling-drop cache policy improves average benchmark scores by 1.7 points and reduces average cost by 27.7%, compared with conventional threshold-triggered summarization.
Sources (2)
- [1]Cache Engineering for AgentsHugging Face Daily Papers · Oct 20, 12:00 AM
Large Language Model (LLM) agents can handle increasingly complex tasks, navigating long trajectories of observations and actions.
Prior work uses context engineering to reduce cache size by trimming or summarizing history.
- [2]Cache Engineering for AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:32 PM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026The Model Swap Workshop: GPT-5.5 vs Kimi vs DeepSeek vs Mistral
- Oct 19, 2026HarnessPro: Recursive Harness Self-Improvement with Pareto Optimization for Performance-Cost Trade-offs
- Oct 19, 2026Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy
- Oct 19, 2026AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments
- Oct 18, 2026System X: An LLM Decision Model That Replies from Its Own KV Cache
- Oct 18, 2026Toward Agentic Retrieval-Augmented Generation for Enterprise Retail Workflows over Heterogeneous Data Sources
