CORD: Conflict-Aware Optimization for Joint RL and Distillation in Agents
Joint reinforcement learning and self-distillation combines task rewards with dense teacher guidance for agent training, but their gradients can partly cancel.
ProofPaper ↗
Key points
- A mixing coefficient controls relative scale without selectively removing opposing components.
- We present CORD, a general framework for conflict-aware optimization of two objectives.
- Gradient diagnostics connect conflict in agent training to lost local optimization progress and motivate selective correction.
- Our analyses explain why protection direction and projection gate must be chosen together, and why the distillation objective and its mixing weight remain consequential after projection.
Sources (1)
- [1]CORD: Conflict-Aware Optimization for Joint RL and Distillation in AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 04:19 PM
Joint reinforcement learning and self-distillation combines task rewards with dense teacher guidance for agent training, but their gradients can partly cancel.
A mixing coefficient controls relative scale without selectively removing opposing components.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 21, 2026One Act, Many Masks: Effect-Marginal Policy Optimization for Long-Horizon Agents
- Oct 20, 2026D-Skill: Distilling Teacher Knowledge through Agent Skills
- Oct 20, 2026SAPD: Learning from Agent Experience without Environment Interaction
- Oct 20, 2026Self-Evolving Agents through Privileged Retrospective Distillation
- Oct 20, 2026AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics
- Oct 20, 2026Training the Whole Agent, Not Just the Model
