ALTK-Evolve: How learned memory tops the τ2-bench leaderboard on customer service tasks without a frontier model

Key points

• Using GPT-4.1, an older, non-reasoning model, a basic customer-service agent reaches 67.5% Pass^4 on τ2-bench retail — meaning it solves the task on all four independent attempts for 67.5% of tasks.1
• ALTK-Evolve builds that playbook using GPT-4.1.1
• τ2-bench (from Sierra) measures how well an agent handles customer-service conversations under real constraints.1
• We run τ2-bench's own implementation of a basic ReAct agent with a two-phase THINK → ACT loop, using gpt-4.1 as the agent model and gpt-5.5 as the customer-simulator model; 4 trials per task.1

Sources (1)

• [1]ALTK-Evolve: How learned memory tops the τ2-bench leaderboard on customer service tasks without a frontier model
  Hugging Face Blog · Oct 20, 03:19 PM

  ALTK-Evolve: How learned memory tops the τ2-bench leaderboard on customer service tasks without a frontier model

  Using GPT-4.1, an older, non-reasoning model, a basic customer-service agent reaches 67.5% Pass^4 on τ2-bench retail — meaning it solves the task on all four independent attempts for 67.5% of tasks.

Extractive summary: sentences quoted from the sources.