Learning from Self-Generated Responses: Reliability--Uncertainty Self-Distillation for Reasoning

Key points

• Reference solutions and question-specific rubrics are not always available to guide the teacher in self-distillation.1
• A large language model's own solution attempts offer an alternative, but their quality is difficult to assess in open-ended tasks without automatic correctness checks, and even correct attempts can involve uncertain token predictions.1
• The same model serves as teacher and student: the teacher receives the question and completed attempt, and the student matches the teacher's token-level predictions using a question-only prompt. \method{} weights each attempt's distillation loss by additively combining judged reliability, estimated from a self-judge's score and confidence, with student predictive uncertainty.1
• Across eleven reasoning benchmarks, \method{} outperforms all five baselines in average accuracy, reaching 70.47% on Qwen3-4B and 76.69% on Qwen3-8B (8.90 and 2.10 percentage points above the base models).1

Sources (1)

• [1]Learning from Self-Generated Responses: Reliability--Uncertainty Self-Distillation for Reasoning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 08:42 PM

  We propose Reliability--Uncertainty Self-Distillation (\method{}) to learn from these attempts.

  Reference solutions and question-specific rubrics are not always available to guide the teacher in self-distillation.

Extractive summary: sentences quoted from the sources.