Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine

Key points

• Converting verifier outcomes into useful policy updates and partial progress into complete task resolution is crucial for terminal agents.1
• We formulate Turn-Boundary Skip-Observation GAE (TB-GAE) to retain token-specific baselines while applying decay across interaction turns.1
• A Warm-up Critic Model, recursive critic handoff, and stage-specific learning rates coordinate the two refinement phases.1
• By recursively coupling verified trajectory rewriting with critic-inheriting policy refinement, each retained stage consolidates discovered behaviors and progressively advances the agent from partial local repairs to strict terminal verification.1

Sources (2)

• [1]Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine
  Hugging Face Daily Papers · Oct 15, 12:00 AM

  We present Terminal-RSI, which alternates Recursive Self-Rewrite (RSR) of verified trajectories with critic-inheriting Recursive Policy-Refine (RPR).

  Converting verifier outcomes into useful policy updates and partial progress into complete task resolution is crucial for terminal agents.

• [2]Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 04:43 PM · same content

Extractive summary: sentences quoted from the sources.