ResearchResearch paperReasoning & Planning · Large Language Models · Training & Scaling2 sources · Oct 21, 2026

TraceForge: Distilling with Your Own Thinking

Using an LLM to generate training data for another is among the most general ways to train one, and it remains common practice.

Some frontier teams have closed that route for their own models: what comes back is an answer, or a short summary, without the detailed reasoning trace behind it.
Recent work recovers the hidden reasoning by replaying encrypted blocks across models.
We take the context and the teacher's answer as input, and generate the reasoning trace ourselves, in the style of the student's own model family.
Training is two-stage, supervised and then adversarial: a generator learns to regenerate the family's own reasoning traces from their context and answer, and a critic learns how real a reasoning trace looks.

[1]TraceForge: Distilling with Your Own Thinking
Hugging Face Daily Papers · Oct 21, 12:00 AM
Using an LLM to generate training data for another is among the most general ways to train one, and it remains common practice.
Some frontier teams have closed that route for their own models: what comes back is an answer, or a short summary, without the detailed reasoning trace behind it.
[2]TraceForge: Distilling with Your Own Thinking
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 12:57 PM · same content

Extractive summary: sentences quoted from the sources.

Before this