Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language Models
We introduce Deep Fusion Memory (DFM), a parametric memory architecture that reduces this construction workload and incorporates learned memory into intermediate hidden states.
ProofPaper ↗
Key points
- Large language models need effective ways to acquire and use external knowledge beyond what they retain from pretraining.
- Continued pretraining risks catastrophic forgetting, while retrieval at inference time can add substantial overhead.
- External parametric memory avoids online retrieval, but constructing its training data requires extensive offline search, and output-level interpolation provides only shallow fusion.
- A causal memory predictor learns to generate memory from the preceding context, using retrieved chunk embeddings from the corpus as supervision.
Sources (1)
- [1]Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 08:24 AM
We introduce Deep Fusion Memory (DFM), a parametric memory architecture that reduces this construction workload and incorporates learned memory into intermediate hidden states.
Large language models need effective ways to acquire and use external knowledge beyond what they retain from pretraining.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 18, 2026Google Open-Sources Harness2, a Recursive Agent Harnessing Framework
- Oct 17, 2026MELD: Recursive Self-Improvement via Hierarchical Coordination for Multi-Agent Scientific Discovery
- Oct 17, 2026Nemotron 3.5 Bolt, Muse Spark Open, and a Fast LFM2.5 24B
- Oct 16, 2026On the Tip of the Tongue: First-Token Cueing as a Behavioral Probe of Recall Failures in LLMs
- Oct 15, 2026openai/codex rust-v0.163.0: 0.163.0
- Oct 15, 2026Accelerating the search for antimicrobial peptides with Google Antigravity