Selective Transfer through Sequence Distillation: From Benchmark Knowledge to Backdoor Behaviour

Key points

• We study whether different hidden teacher properties survive the same distillation channels.1
• Specifically, we compare two properties: benchmark-associated knowledge and refusal backdoors, and show that distillation treats them very differently.1
• Teachers fine-tuned on benchmark data successfully pass that knowledge to students through all four distillation channels we test, including when students are trained only on teacher-generated number sequences containing no natural language or domain content whatsoever.1
• Refusal backdoors tell the opposite story: backdoored teachers achieve near-perfect triggered refusal, yet sequence-distilled students inherit essentially none of this behaviour.1

Sources (1)

• [1]Selective Transfer through Sequence Distillation: From Benchmark Knowledge to Backdoor Behaviour
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 12:56 PM

  Knowledge distillation is widely used to transfer capabilities from teacher models to student models, but recent work shows that it can also transfer unintended teacher properties.

  We study whether different hidden teacher properties survive the same distillation channels.

Extractive summary: sentences quoted from the sources.