D-Skill: Distilling Teacher Knowledge through Agent Skills

Key points

• Frontier language-model agents excel at complex tasks, but high inference costs make transferring their capabilities to smaller models essential.1
• Traditional knowledge distillation achieves this through parameter updates, requiring access to the student's weights and additional training compute while encoding the transferred knowledge implicitly in the parameters.1
• D-Skill constructs the skill entirely offline from fixed trajectories, without labels, further environment interaction, or weight updates after rollout collection.1
• Our results demonstrate that natural-language skill distillation offers a performant, interpretable, and efficient alternative to weight-based capability transfer.1

Sources (1)

• [1]D-Skill: Distilling Teacher Knowledge through Agent Skills
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 08:52 PM

  We introduce D-Skill, a framework that treats the teacher as a black box and distills its behavioral advantage into an interpretable natural-language skill by analyzing paired rollouts and identifying behavioral divergences and agreements.

  Frontier language-model agents excel at complex tasks, but high inference costs make transferring their capabilities to smaller models essential.

Extractive summary: sentences quoted from the sources.