CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation

Key points

• Cross-tokenizer on-policy distillation (OPD) requires aligning teacher and student sequences before their scores can be compared.1
• Decoded-text alignment provides such correspondence at the chunk level, but leaves a distinct problem unresolved: credit alignment.1
• For aligned chunks, CAST redistributes each chunk's advantage across student tokens using predictive uncertainty and local teacher-signal smoothness, emphasizing learning opportunities supported by locally consistent teacher feedback while preserving the chunk's total advantage.1
• Across six mathematics, science, and code benchmarks, CAST improves the average over the strongest evaluated baseline by $2.45$ and $1.32$ points for cross-tokenizer transfer from Qwen3-8B to Mistral-7B and Llama-3.1-8B, respectively, and by $2.10$ points for shared-tokenizer transfer to Qwen3-1.7B.1

Sources (1)

• [1]CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 09:03 AM

  We propose CAST (Chunk-aware credit Assignment with Selective Token Anchoring), a shift-or-stay framework that addresses both issues.

  Cross-tokenizer on-policy distillation (OPD) requires aligning teacher and student sequences before their scores can be compared.

Extractive summary: sentences quoted from the sources.