DiaLong: Benchmarking Reminiscence in Long-Context Dialogues
How well can transformer-based Large Language Models (LLMs) retain and recall memories in long contexts of dialogue?
ProofPaper ↗
Key points
- We present DiaLong, a long-context dialogue dataset and accompanying memory task that standardizes evaluation in this space.
- Our selected memory task is fact recall: classify 40 candidate facts about a dialogue as true or false.
- In testing both the latest LLMs and proposed memory enhancements on DiaLong, we demonstrate that more work needs to be done to enable LLMs with better memory performance.
- By enhancing memory capabilities, LLMs can improve in maintaining coherence, relevance, and human-like interaction over extended conversation.
Sources (1)
- [1]DiaLong: Benchmarking Reminiscence in Long-Context DialoguesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 06:47 PM
How well can transformer-based Large Language Models (LLMs) retain and recall memories in long contexts of dialogue?
We present DiaLong, a long-context dialogue dataset and accompanying memory task that standardizes evaluation in this space.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 18, 2026vllm-project/vllm v0.32.0
- Oct 17, 2026deepseek-ai/DeepSeek-V4.1-Flash-1017
- Oct 16, 2026Prefix-Tuning for Arbitrary Output Sequences on Pretrained Transformers
- Oct 16, 2026Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference
- Oct 16, 2026Welcome Beam: a 501B open model you can own, customize, and scale
- Oct 16, 2026Language Models Learn to Improve Language Models