All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents

Key points

• Large language models increasingly rely on external memory to answer queries grounded in long-horizon histories such as multi-session dialogues and lengthy documents.1
• A unified memory policy actor then rolls out complete interaction trajectories over this shared evidence state.1
• A separate trajectory-conditioned process reward model is trained through evidence-grounded supervised fine-tuning and trajectory-level DPO to assign process scores to the collected trajectories.1
• Across long-context memory benchmarks, AiO-Mem consistently improves answer accuracy over both static memory systems and RL-trained memory agents.1

Sources (1)

• [1]All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 03:02 AM

  To address this, we propose All-in-One Memory (AiO-Mem), which organizes these operations as transitions over a shared evidence state and directly evaluates the semantic quality of each action in a unified reward space.

  Large language models increasingly rely on external memory to answer queries grounded in long-horizon histories such as multi-session dialogues and lengthy documents.

Extractive summary: sentences quoted from the sources.