Learning to Compute Is Not Learning to Retain or Reuse in Looped Transformers

Key points

• Retention depends on what training asks for: supervising one randomly chosen position per example, instead of always the last, keeps 99.8%, and randomizing the number of training loops also helps; truncating gradients, as in progressive training against overthinking, does not.1
• The computation itself is exact: position $k$ is computed at loop $k$, and swapping its loop-$k$ hidden vector for the one from another input redirects every later position to that input's exact continuation (237/237 pairs in each of ten models).1
• A released recurrent maze network trained to avoid overthinking behaves the same way.1
• Learning to compute is thus not learning to retain, and avoiding overthinking does not make a state reusable when its answer first appears.1

Sources (1)

• [1]Learning to Compute Is Not Learning to Retain or Reuse in Looped Transformers
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 08:19 AM

  Looped transformers compute by applying one block repeatedly, so they can run for more loops than they were trained with.

  Retention depends on what training asks for: supervising one randomly chosen position per example, instead of always the last, keeps 99.8%, and randomizing the number of training loops also helps; truncating gradients, as in progressive training against overthinking, does not.

Extractive summary: sentences quoted from the sources.