ActionOPD: Distilling Decisions, Not Tokens
We introduce ActionOPD, an action-level on-policy distillation method motivated by maximizing expected teacher-action likelihood at student-encountered states.
ProofPaper ↗
Key points
- Distilling capable language-model agents into smaller models requires aligning the student's behavior with the teacher's.
- This raises a fundamental question: at which states and at what granularity should this alignment be performed?
- We argue that semantic actions provide a natural unit of supervision for agent distillation.
- Through policy optimization, ActionOPD aligns the student's action-level behavior with the teacher's decisions without requiring reproduction of teacher-written reasoning or access to teacher token probabilities.
Sources (2)
- [1]ActionOPD: Distilling Decisions, Not TokensarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 08:59 AM
We introduce ActionOPD, an action-level on-policy distillation method motivated by maximizing expected teacher-action likelihood at student-encountered states.
Distilling capable language-model agents into smaller models requires aligning the student's behavior with the teacher's.
- [2]ActionOPD: Distilling Decisions, Not TokensHugging Face Daily Papers · Oct 14, 12:00 AM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026DTDD: Divergence-Triggered Dynamic Distillation for Reliable On-Policy Supervision
- Oct 13, 2026Outcome-Routed Distillation and Reinforcement Learning for Diffusion Language Models
- Oct 13, 2026Learning from Self-Generated Responses: Reliability--Uncertainty Self-Distillation for Reasoning
- Oct 13, 2026Demystifying On-Policy Distillation: Where Does the Teacher's Advantage Live?
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
- Oct 13, 2026Agent-Reflex: Teaching Language Agents to Act Reflectively
