Behavioral Compositionality of Sparse Autoencoder Representations

Key points

• Sparse autoencoders (SAEs) represent model activations with sparse, linearly decoded features, but the downstream effects of those features need not compose additively.1
• Under independent feature subset sampling, population SAR is exactly the normalized energy of interactions above that order and bounds coalition ordering error for the optimal low order approximation.1
• To distinguish compositionality associated with the SAE representation from that induced by downstream computation, we compare learned SAE coordinates with sum preserving random rotations of the same selected subspace and find lower first order residuals for the learned coordinates across all 18 model and behavior settings.1
• In a combined steering experiment, pairwise modeling raises rank fidelity from 0.5874 to 0.7790 and reduces ordering error from 0.1444 to 0.0606.1

Sources (2)

• [1]Behavioral Compositionality of Sparse Autoencoder Representations
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 04:58 AM

  We introduce SAE Additivity Residuals (SAR), which measure the behavioral variation left unexplained above a chosen interaction order.

  Sparse autoencoders (SAEs) represent model activations with sparse, linearly decoded features, but the downstream effects of those features need not compose additively.

• [2]Behavioral Compositionality of Sparse Autoencoder Representations
  Hugging Face Daily Papers · Oct 15, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.