Do Conversational Agents Need Explicit Memory? A Controlled Comparison with Raw-History Retrieval

Key points

• This study compares six state-of-the-art explicit memory systems with raw-history RAG on LongMemEval-S and BEAM-100K under original and controlled retrieval settings.1
• We find only localized gains for particular question types and tight run-time context budgets, with no consistent advantage over strong raw-history RAG baselines.1
• Oracle retrieval shows that some systems fail to preserve answer-supporting information during explicit memory construction, while others preserve it but fail to retrieve it effectively, despite construction costs reaching hours per conversation.1
• Our findings suggest that while many existing memory systems for AI agents emphasize increasingly elaborate explicit memory construction, they are often evaluated without strong controlled comparisons to raw-history RAG and may underemphasize retrieval.1

Sources (1)

• [1]Do Conversational Agents Need Explicit Memory? A Controlled Comparison with Raw-History Retrieval
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 12:44 PM

  Explicit memory systems transform conversation history into persistent representations intended to improve future access, often at substantial construction cost.

  This study compares six state-of-the-art explicit memory systems with raw-history RAG on LongMemEval-S and BEAM-100K under original and controlled retrieval settings.

Extractive summary: sentences quoted from the sources.