PA1-OPD: One-Step Prefix-Augmented On-Policy Distillation for Language Model Pretraining
Knowledge distillation (KD) improves language-model pretraining by matching teacher distributions at corpus prefixes, but provides no direct supervision at contexts induced by student predictions.
ProofPaper ↗
Key points
- We introduce PA1-OPD, which augments corpus-path distillation with teacher feedback after one student-sampled transition per selected prefix.
- This expands the supervised states while retaining next-token prediction and corpus-path KD.
- Data-anchored roots support parallel sampling, and differentiable tree attention shares prefix computation without changing inference.
- These findings support local student-induced supervision as a complement to pretraining distillation.
Sources (1)
- [1]PA1-OPD: One-Step Prefix-Augmented On-Policy Distillation for Language Model PretrainingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 05:23 AM
Knowledge distillation (KD) improves language-model pretraining by matching teacher distributions at corpus prefixes, but provides no direct supervision at contexts induced by student predictions.
We introduce PA1-OPD, which augments corpus-path distillation with teacher feedback after one student-sampled transition per selected prefix.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026ActionOPD: Distilling Decisions, Not Tokens
- Oct 13, 2026Outcome-Routed Distillation and Reinforcement Learning for Diffusion Language Models
- Oct 13, 2026Demystifying On-Policy Distillation: Where Does the Teacher's Advantage Live?
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
- Oct 13, 2026Simplifying and Improving On-Policy Distillation for Few-Step Diffusion
- Oct 12, 2026JustOPD: On-Policy Distillation from Autoregressive to Diffusion Language Models