Language Models Share a Readout Bottleneck in Working Memory
Large language models (LLMs) reproduce a human working-memory signature on the N-back task: performance declines as memory load N increases, and errors concentrate on lures, items that repeat the current stimulus at a neighboring but incorrect serial position.
ProofPaper ↗
Key points
- We ask whether this shared behavioral signature reflects a shared mechanism.
- Linear probes decode near-perfectly whether a matching letter occurred at the target or at a lure position, even on trials where the model false-alarms to the lure, although its own decisions barely distinguish the two.
- In larger models the task is spread over more heads, and removing interference helps more.
- In minimal transformers, precision limits a single attention layer, but a second layer removes this constraint.
Sources (1)
- [1]Language Models Share a Readout Bottleneck in Working MemoryarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 03:32 AM
Large language models (LLMs) reproduce a human working-memory signature on the N-back task: performance declines as memory load N increases, and errors concentrate on lures, items that repeat the current stimulus at a neighboring but incorrect serial position.
We ask whether this shared behavioral signature reflects a shared mechanism.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026Optimal Hydra: Model Heterogeneous Federated Learning for LLMs using Data-Informed Hydra-Nets
- Oct 14, 2026mistralai/Timestral-209M
- Oct 14, 2026Who's Behind the Text? Robust Model Authorship Attribution
- Oct 14, 2026Why don't Graph Transformers attend to the same token?
- Oct 14, 2026Sparse Autoencoders Are a Shared Coordinate System for Symbolic Matching
- Oct 13, 2026Narrowing the Collaboration Gap, Probably