Learning When to Abort in Transformer Planning: A Mechanistic Study

Key points

• Planning does not by itself imply the ability to recognise errors, but recognising errors can improve planning capacity.1
• To analyze the inner mechanism, we follow the approach of Wang et al. (2024) and abstract planning as path-finding in graphs.1
• In this setting, we train the model to detect the error of invalid transitions, i.e., after the model generates a next node that is not adjacent with the current node, the model should then generate a special abort token x.1
• In trained models, we demonstrate that invalid-transition detection increases previous-node attention, and most of the valid--invalid separation locates in the feed-forward network's output.1

Sources (1)

• [1]Learning When to Abort in Transformer Planning: A Mechanistic Study
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 02:33 PM

  In this paper, we study this phenomenon in Transformer planning.

  Planning does not by itself imply the ability to recognise errors, but recognising errors can improve planning capacity.

Extractive summary: sentences quoted from the sources.