From Chain-of-Thought to Loops: Non-Autoregressive Latent Reasoning via Looped Transformers

Key points

• Chain-of-thought (CoT) reasoning often improves language-model performance by giving models additional computation before answering.1
• Latent reasoning replaces these tokens with compact continuous states, but most autoregressive latent-reasoning methods retain a left-to-right dependency among latent vectors.1
• Training uses continuous representations derived from explicit CoT together with a final-answer prediction loss and likelihood-based supervision of the latent states.1
• This design replaces serial thought generation with parallel latent-slot refinement while retaining probabilistic latent modeling and autoregressive answer decoding.1

Sources (1)

• [1]From Chain-of-Thought to Loops: Non-Autoregressive Latent Reasoning via Looped Transformers
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 08:21 AM

  We introduce LLoCoT: a looped latent-reasoning framework that replaces left-to-right latent generation with iterative refinement of a compact latent workspace.

  Chain-of-thought (CoT) reasoning often improves language-model performance by giving models additional computation before answering.

Extractive summary: sentences quoted from the sources.