Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents

Key points

• While LLM-based agent systems have demonstrated remarkable proficiency in reasoning and solving complex real-world tasks, they also come with substantial token consumption and inference latency, limiting their practical deployment.1
• To enhance the efficiency of agent systems, recent efforts have focused on transferring agentic capabilities from large-scale models to small language models (SLMs) via supervised fine-tuning-based distillation.1
• Nevertheless, as interaction trajectories grow longer, involving extended reasoning-action chains and extensive environmental feedback, distilled student agents tend to experience progress stagnation: becoming trapped in unproductive loops characterized by persistent failed strategies and difficulty making meaningful progress.1
• Extensive experiments involving 8 SLMs, ranging from 0.6B to 8B parameters, on both in-domain and out-of-domain benchmarks, demonstrate that ProD substantially mitigates progress stagnation while enhancing both the success rates and task-completion efficiency of distilled SLM agents.1

Sources (2)

• [1]Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 02:31 AM

  To address this issue, we propose Progress-Aware Distillation (ProD), which explicitly detects and penalizes stagnation in student-generated trajectories.

  While LLM-based agent systems have demonstrated remarkable proficiency in reasoning and solving complex real-world tasks, they also come with substantial token consumption and inference latency, limiting their practical deployment.

• [2]Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents
  Hugging Face Daily Papers · Oct 17, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.