Not Every User Says It All: Training Tool-Using Agents Beyond Ideal Simulated Users

Key points

• Tool-using agents are trained and evaluated almost exclusively with strong simulated users who state every detail clearly, yet real users are often weaker.1
• Weak users lower the success of every policy, and training with them does not help: no policy trained with weak users, through fixed masks, mixtures or a curriculum, serves weak users better than the strong-trained policy beyond run-to-run noise, and prompting the agent to ask more does not help either.1
• We propose StaGPO (State-Grouped Policy Optimization): the agent writes what the task needs, what is missing and how it is being filled, and GRPO uses this state to compare turns of different rollouts that reach the same situation, assigning step-level credit.1
• On tau2-bench it scores highest with weak users for both models, by a small margin for Qwen3.5-4B.1

Sources (1)

• [1]Not Every User Says It All: Training Tool-Using Agents Beyond Ideal Simulated Users
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 10:41 PM

  We construct weak users by masking sentences of a simulated user's replies, holding the task and the user's knowledge fixed, and cross-evaluate agents trained with strong and weak users on tau2-bench and ToolWOZ.

  Tool-using agents are trained and evaluated almost exclusively with strong simulated users who state every detail clearly, yet real users are often weaker.

Extractive summary: sentences quoted from the sources.