Remember Less, Adapt Longer: Bounded-State In-Context Reinforcement Learning with Recurrent Summary Memory

Key points

• For agents operating continually, existing approaches using memory trade off bounded cost against access to past experience.1
• Fixed-size memories bound storage but restrict access: sliding windows discard older steps, while recurrent networks compress even recent interactions into a hidden state.1
• We introduce Recurrent Summary Memory (RSM), a Transformer policy that attends exactly to a short working buffer of recent interactions and carries a fixed-capacity summary of earlier experience.1
• On long-running partially observable tasks, frozen RSM policies sustain their adapted performance for eight times the training horizon with about 1% of full-history memory, ending ahead of every evaluated baseline while growing memories deteriorate.1

Sources (1)

• [1]Remember Less, Adapt Longer: Bounded-State In-Context Reinforcement Learning with Recurrent Summary Memory
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 04:13 AM

  In-context reinforcement learning (ICRL) enables agents to adapt to new tasks without parameter updates, relying on memory to retain what they learn from experience.

  For agents operating continually, existing approaches using memory trade off bounded cost against access to past experience.

Extractive summary: sentences quoted from the sources.