Ask to Distill: Learning to Synthesize Data for On-Policy Distillation

Key points

• On-policy distillation (OPD) can succeed with surprisingly few training questions, motivating a view of questions as carriers of distributional knowledge transfer.1
• How can we assess whether a newly generated question is valuable for distillation, and learn to generate better questions---without access to its correct answer?1
• A student-initialized synthesizer is optimized using a distributional surrogate for question value, computed from a frozen student--teacher pair without reference answers, correctness rewards, or per-candidate student updates.1
• Signal analyses further show that optimized questions can elicit denser supervision and more total signal from shorter student responses, supporting elicited distributional supervision as an objective for synthetic distillation data.1

Sources (2)

• [1]Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
  Hugging Face Daily Papers · Oct 20, 12:00 AM

  We introduce Ask-to-Distill (\method), a framework that treats synthetic questions as distribution elicitors and optimizes the teacher supervision they expose along student-generated trajectories.

  On-policy distillation (OPD) can succeed with surprisingly few training questions, motivating a view of questions as carriers of distributional knowledge transfer.

• [2]Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 01:00 AM · same content

Extractive summary: sentences quoted from the sources.