Self-Distillation for Supervised Fine-Tuning

Key points

• Supervised fine-tuning treats the expert demonstration or gold response as a one-hot target, suppressing valid alternative continuations that the model would otherwise assign probability to.1
• GCD uses one model as both teacher and student: the teacher conditions on a prompt containing the gold response in context, the student conditions on the original prompt as at inference, and the student is trained to minimize the forward KL divergence to the teacher's next-token distribution at each gold response token.1
• This lets the model use its prior knowledge to teach itself the new task: correctness is anchored by the gold response, while the soft target preserves plausible alternatives.1
• Across various training tasks on Qwen and Llama models, GCD improves new-task accuracy over SFT and generally reduces degradation on general benchmarks.1

Sources (1)

• [1]Self-Distillation for Supervised Fine-Tuning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 07:35 PM

  We propose Gold-Conditioned Distillation (GCD), a method that turns expert demonstrations into model-native supervision.

  Supervised fine-tuning treats the expert demonstration or gold response as a one-hot target, suppressing valid alternative continuations that the model would otherwise assign probability to.

Extractive summary: sentences quoted from the sources.