A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt

Key points

• The bound holds for finite-depth pre-LN softmax transformers with bounded attention logits, under one additional condition: the pre-norm activations stay away from zero.1
• This is what we call the conditional no-hair theorem: given the text generated so far, the prompt's influence on the next token vanishes as t grows, so any property of the prompt that survives, such as its language, must be carried by the text itself.1
• An exactly solvable linear relay model shows that the bound cannot be improved and reproduces the rise of the local exponent towards 1 seen in untrained networks.1
• Cancelling the dilution with a logit bias stops the decay early in training and reverses it after, so dilution is necessary for the forgetting we measure; untrained networks and the solvable model show it is also sufficient.1

Sources (2)

• [1]A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt
  Hugging Face Daily Papers · Oct 16, 12:00 AM

  We ask how much an autoregressive transformer retains of its prompt after generating t tokens.

  The bound holds for finite-depth pre-LN softmax transformers with bounded attention logits, under one additional condition: the pre-norm activations stay away from zero.

• [2]A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 07:03 PM · same content

Extractive summary: sentences quoted from the sources.