P2N: Reusing Deep Representations for Greater Effective Depth

Key points

• Mathematical problem solving and autonomous agents are increasingly important applications of language models, demanding stronger reasoning capabilities.1
• Computational depth is a key resource for such reasoning, but adding model layers increases both training and inference costs, while test-time scaling incurs substantial additional inference computation.1
• This recurrence extends effective computational depth across tokens, while introducing only a single vector addition per decoding step.1
• We train the recurrence using parallel Jacobi updates restricted to a middle-layer core.1

Sources (1)

• [1]P2N: Reusing Deep Representations for Greater Effective Depth
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 07:32 AM

  We introduce P2N (Previous to Next), which reuses the previous token's deep representations, already available during generation, to increase effective computational depth with negligible decoding overhead.

  Mathematical problem solving and autonomous agents are increasingly important applications of language models, demanding stronger reasoning capabilities.

Extractive summary: sentences quoted from the sources.