Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction

Key points

• Unlike prior training-based approaches that fine-tune an LLM to generate revisions or feedback, we learn where to intervene in failed rollouts and determine how to redirect them.1
• To choose where to intervene, we train a 130K-parameter MLP head on a frozen encoder using only final-answer correctness.1
• For redirection, we find that under a fixed generation budget, the tested inline injections fail to reliably redirect reasoning, whereas presenting the same instruction in a new user turn changes the model's course.1
• The resulting correction framework, UPTURN (User-Prompt TURN injection at a learned position), requires no LLM fine-tuning: it retains a selected prefix, injects an instruction as a new user turn, and regenerates the suffix.1

Sources (1)

• [1]Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 02:37 PM

  Large language models struggle to reliably identify and correct errors in their own reasoning.

  Unlike prior training-based approaches that fine-tune an LLM to generate revisions or feedback, we learn where to intervene in failed rollouts and determine how to redirect them.

Extractive summary: sentences quoted from the sources.