Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility

Key points

• However, differences in design assumptions, evaluation objectives, and experimental settings complicate conclusions about their effectiveness.1
• We present a critical survey of 138 studies on SAEs in natural language processing.1
• Through comparative analyses of published results, we reconcile apparent disagreements, identify unresolved questions, and propose directions for more rigorous evaluation.1

Sources (1)

• [1]Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 12:56 PM

  Sparse autoencoders (SAEs) have emerged as a promising approach to mechanistic interpretability by decomposing language-model activations into sparse, potentially interpretable features.

  However, differences in design assumptions, evaluation objectives, and experimental settings complicate conclusions about their effectiveness.

Extractive summary: sentences quoted from the sources.