Representations in Motion: Layer-wise Rotation and Trajectory Divergence in LLMs

Key points

• In this work, we investigate the layer-wise evolution of representations of factuality, using true and false statements when the model is prompted to either lie or to be honest.1
• We find that such layer-wise rotation also occurs in a task that requires the model to take different visual perspectives in the 3D world, suggesting that the rotation phenomenon might be indicative of manipulating conceptual spaces in general, rather than unique to factuality.1
• We show that it is possible to make strong predictions about the model's target task and behavior from trajectories alone.1
• Finally, we show that both rotation and divergent trajectories occur in more naturalistic settings in which models are not overtly prompted to "lie" but rather report false statements in the course of completing a different task.1

Sources (1)

• [1]Representations in Motion: Layer-wise Rotation and Trajectory Divergence in LLMs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 10:28 PM

  Recent work suggests that activation dynamics--how representations change during inference--might be more revealing about language model behavior than static readouts from individual layers.

  In this work, we investigate the layer-wise evolution of representations of factuality, using true and false statements when the model is prompted to either lie or to be honest.

Extractive summary: sentences quoted from the sources.