Curriculum Learning Teaches LLMs Long-Horizon Implicit Reasoning, Up To A Limit

Key points

• Implicit reasoning (reasoning within a single forward pass) is useful during CoT generation, as the model can "search ahead" to decide the correct next move without explicit search.1
• Models with weak implicit reasoning are more likely to commit to wrong paths and must attempt to recover via backtracking, while models with strong implicit reasoning identify the correct proof strategy directly, shortening the CoT.1
• We investigate whether curriculum learning can extend this depth by fine-tuning on examples with progressively increasing vocabulary size and lookahead (the number of steps a model must search ahead to identify the correct next move).1
• Finally, the capability acquired on synthetic data transfers to natural language reasoning, with curriculum-trained models outperforming models trained without curriculum at matched compute on 11 natural language reasoning benchmarks.1

Sources (1)

• [1]Curriculum Learning Teaches LLMs Long-Horizon Implicit Reasoning, Up To A Limit
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 12:32 AM

  LLMs "reason" by externalizing intermediate steps as tokens through chain-of-thought (CoT), yet even frontier reasoning models hit abrupt performance cliffs on long-horizon tasks.

  Implicit reasoning (reasoning within a single forward pass) is useful during CoT generation, as the model can "search ahead" to decide the correct next move without explicit search.

Extractive summary: sentences quoted from the sources.