MemCut: Every Secret Leaves a Trail in the Long-Term Memory of LLM Agents
To evaluate this capability, we introduce MemCut, a systematic benchmark for user-requested forgetting.
ProofPaper ↗
Key points
- Long-term memory has become a crucial component of large language model (LLM) agents, with most research focusing on retrieving relevant information accurately and efficiently.
- Native recovery measures reader success through the system's retrieval, whereas residual recovery measures the same reader's success when directly given surviving objects traced to annotated disclosures or evidence.
- We establish three reference deletion baselines to test how far existing retrievers and write models support forgetting across five memory designs and four LLMs. Disclosure removal reduces native recovery far more than residual recovery.
- LLM-guided deletion of ten additional turns from twenty retrieval candidates leaves 39-76% residual recovery.
Sources (1)
- [1]MemCut: Every Secret Leaves a Trail in the Long-Term Memory of LLM AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 07:23 AM
To evaluate this capability, we introduce MemCut, a systematic benchmark for user-requested forgetting.
Long-term memory has become a crucial component of large language model (LLM) agents, with most research focusing on retrieving relevant information accurately and efficiently.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
- Oct 15, 2026JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs
- Oct 15, 2026GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations
- Oct 15, 2026Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries
- Oct 15, 2026Sci$Π$vot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents
- Oct 13, 2026Plan or Retrieve First? A Scale Study of Tool Discovery Architectures