Shaped SAEs: Expanding the Toolkit for Mechanistic Interpretability
Sparse autoencoders (SAEs) seek to decompose the dense activations of large language models (LLMs) into sparse, potentially interpretable features using a learned overcomplete dictionary.
ProofPaper ↗
Key points
- In this work, we ask whether it is possible to enhance the semantic selectivity of the learned dictionary atoms by linearly shaping the dense activations prior to learning an SAE.
- We explore rank-reducing transformations based on the recently introduced J-space corresponding to the averaged Jacobian from the current layer to the LLM output, projecting activations onto the "live" subspace spanned by eigenmodes of the Jacobian with larger singular values (which we expect to be more semantically relevant).
- We also introduce a new approach for training SAEs in which (a) an affine shaping transformation is learned along with the dictionary atoms, (b) the atom selection mechanism is based on the TopK projections along the one-dimensional subspace spanned by the atoms, and (c) reconstruction is carried out using the same atoms.
- We call this approach a Matched Filter SAE (MF-SAE) based on analogies with communication theory.
Sources (1)
- [1]Shaped SAEs: Expanding the Toolkit for Mechanistic InterpretabilityarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 10:58 PM
Sparse autoencoders (SAEs) seek to decompose the dense activations of large language models (LLMs) into sparse, potentially interpretable features using a learned overcomplete dictionary.
In this work, we ask whether it is possible to enhance the semantic selectivity of the learned dictionary atoms by linearly shaping the dense activations prior to learning an SAE.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Evaluating Agents on Highly Subjective Tasks
- Oct 19, 2026Introducing Clairic, a new System One class of extraction models
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
- Oct 19, 2026Rethinking LLM Fine-Tuning via Weight Space Reparameterization: Preserving Safety during Downstream Adaptation
- Oct 19, 2026From Black Box to Controller: Inference-Time Steering of LLM via Feature Amplification
- Oct 17, 2026A Cross-Domain Evaluation of Prompting Techniques for Large Language Models