Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules
To address this gap, we introduce Titan-Llama, a post-training method for retrofitting attention-based language models with a hierarchical memory architecture that updates parameters at test-time.
ProofPaper ↗
Key points
- Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference requires increasingly large KV caches.
- Efficient recurrent alternatives reduce the asymptotic complexity of inference, but their sequential dependence limits token-wise parallelism during training, making them harder to scale.
- Titan-Llama applies a blockwise attention mask and preserves full-fidelity attention over recent tokens while compressing historical context into recurrent, fixed-size neural memory modules (NMMs).
- Together, these results demonstrate that incorporating memory architecture mechanisms that utilize test-time training effectively modulates long-context behavior of pre-trained models, offering a practical route to adapting pre-trained transformers for efficient long-context inference.
Sources (1)
- [1]Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory ModulesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 09:23 PM
To address this gap, we introduce Titan-Llama, a post-training method for retrofitting attention-based language models with a hierarchical memory architecture that updates parameters at test-time.
Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference requires increasingly large KV caches.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
- Oct 13, 2026Gemini 4 Argon: Built to solve your hardest problems
- Oct 13, 2026Flama: 8 years building a Python framework where ML models and LLMs are first-class, now written up on arXiv [P]
- Oct 13, 2026I built a pytorch library of pretrained models for 3D point clouds following timm standards [P]
- Oct 13, 2026Copy-Paste Language Models
- Oct 12, 2026MLAfication: Efficiently Enabling High-Fidelity MLA for Modern LLMs