JustOPD: On-Policy Distillation from Autoregressive to Diffusion Language Models
On-policy distillation (OPD) is a post-training approach for large language models (LLMs) that enables a student model to learn a teacher model's distribution from sequences sampled from the student's own policy.
ProofPaper ↗
Key points
- Despite its promising capability, OPD for diffusion language models (DLMs) remains underexplored and existing studies primarily focuses on DLM-to-DLM distillation.
- The limited availability of strong teacher DLMs and the flexible generation order of DLMs impose additional technical challenges as well.
- This alignment enables effective distillation from ARM teachers while preserving the DLM's flexible generation-order capability at inference time.
- Experiments using Qwen3-14B as the teacher and Dream-7B as the student demonstrate that JustOPD consistently improves mathematical reasoning performance over the base DLM across multiple divergence choices.
Sources (1)
- [1]JustOPD: On-Policy Distillation from Autoregressive to Diffusion Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 09:02 PM
On-policy distillation (OPD) is a post-training approach for large language models (LLMs) that enables a student model to learn a teacher model's distribution from sequences sampled from the student's own policy.
Despite its promising capability, OPD for diffusion language models (DLMs) remains underexplored and existing studies primarily focuses on DLM-to-DLM distillation.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026unslothai/unsloth v0.1.906-beta: Browser Use + Qwen-Image-2.1-Turbo
- Oct 12, 2026A Production-Grade, High-Efficiency Agentic RL Framework for Frontier LLMs with Validated Recipes
- Oct 12, 2026Omni-Modal LLMs Know More Than They Say
- Oct 11, 2026The Mathematical Collective Unconscious: Jungian Archetypes within the Latent Space
- Oct 11, 2026GA-SVD: Generation-aware Singular Value Decomposition for LLM Compression
- Oct 10, 2026SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usability