Augment the Cache, Replace the Activation: A 2x2 Study of Internal-State Sharing

Key points

• Work on internal channels avoids that cost, framing the problem as transfer: agent A sends information to agent B.1
• We reframe this problem as sharing: an agent publishes its internal state once into a pool, and every peer reads that pool through a single mechanism fitted to no sender in particular, so adding an agent adds a publisher and no new bridge.1
• Whereas the carrier of the internal state and the fusion mechanism are usually chosen independently in the literature, our main finding is that these two choices are coupled; the deciding factor is where the carrier enters the receiver's computation.1
• We find that the key-value (KV) cache must be augmented rather than swapped in: the receiver cross-attends over a peer's cache while keeping its own, and with only a small query-side adapter trained, matches the accuracy it gets from reading the source document.1

Sources (1)

• [1]Augment the Cache, Replace the Activation: A 2x2 Study of Internal-State Sharing
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 03:04 PM

  Multi-agent systems built from frozen language models communicate through text, which discards the internal state the sender already computed and makes the receiver pay to rebuild it.

  Work on internal channels avoids that cost, framing the problem as transfer: agent A sends information to agent B.

Extractive summary: sentences quoted from the sources.