ResearchResearch paperLarge Language Models · Reinforcement Learning · Training & Scaling1 source · Oct 10, 2026

Beyond Outcome Feedback: Trajectory-Quality-Guided Experience-based Learning

To address this limitation, we propose TQG-Exp, a trajectory-quality-guided experience-based learning framework that introduces process feedback.

Experience-based learning enables large language model (LLM) agents to improve over time by reusing knowledge from past executions.
However, existing methods typically rely on final task outcomes as feedback for experience learning, overlooking the execution process and limiting their ability to evaluate and optimize experience based on trajectory quality.
Specifically, we introduce Trajectory Quality Loss (TQL), which jointly considers task completion and process quality to provide a learning signal beyond final task outcomes.
Experiments on ALFWorld, WebShop, and ScienceWorld with two backbone LLMs show that TQG-Exp outperforms existing experience-based methods, improving Avg.@3 over the strongest baseline by 4.50 and 8.55 percentage points on DeepSeek-V4-Flash and Qwen3.5-Flash, respectively.

[1]Beyond Outcome Feedback: Trajectory-Quality-Guided Experience-based Learning
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 12:50 PM
To address this limitation, we propose TQG-Exp, a trajectory-quality-guided experience-based learning framework that introduces process feedback.
Experience-based learning enables large language model (LLM) agents to improve over time by reusing knowledge from past executions.

Extractive summary: sentences quoted from the sources.

Before this