Sparse autoencoders

Timeline

1. Oct 16, 2026 · Opinion / analysis · 1 source
   Understanding How Neural Networks Learn (with interactive visualizations)
   This essay rewrites fundamental assumptions about how neural networks learn and connect, with some of the best interactive visualizations I've ever seen.
2. Oct 16, 2026 · Research paper · 1 source
   Beyond SAEs: Consistent Concept Discovery with Sparse Coding
   We propose Consistent Sparse Dictionary Learning (ConSDL), which replaces the standard linear encoder with per-sample sparse coding.
3. Oct 16, 2026 · Research paper · 1 source
   MS-Eval: Benchmarking Sub-Agent Delegation and Orchestration in MCP-Intensive Working Scenarios
   The Main-Agent Delegation and Sub-Agent Execution (MAD-SAE) paradigm addresses these issues by delegating sub-tasks to specialized sub-agents for parallel execution in isolated contexts, but its delegation and orchestration quality remains unquantified.
4. Oct 15, 2026 · Research paper · 1 source
   Native Geometry vs. Linear Recoverability: How Alignment Changes Platonic Scaling
   Using matched Legacy Survey and Hyper Suprime-Cam (HSC) embeddings across five model families, we find that Ridge alignment strengthens the relationship between model size and cross-dataset neighbourhood correspondence relative to native cosine geometry.
5. Oct 15, 2026 · Research paper · 2 sources
   Behavioral Compositionality of Sparse Autoencoder Representations
   We introduce SAE Additivity Residuals (SAR), which measure the behavioral variation left unexplained above a chosen interaction order.
6. Oct 15, 2026 · Research paper · 1 source
   Representation Redistribution in Sparse Autoencoders
   Sparse autoencoders (SAEs) decompose language model activations into sparse features, but prompt transformations designed to preserve the underlying request can redistribute activity across them.
7. Oct 14, 2026 · Research paper · 1 source
   Mind the Tail: Data Selection Through Sparse Concepts
   Data selection reallocates training support across visual concepts, including objects, attributes, color, texture, and backgrounds.
8. Oct 14, 2026 · Incident · 1 source
   What behaviors does a model actually learn in RL? [P]
   I've been working on unsupervised tooling to discover what behaviors a model learns in post-training.
9. Oct 14, 2026 · Research paper · 1 source
   Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility
   Sparse autoencoders (SAEs) have emerged as a promising approach to mechanistic interpretability by decomposing language-model activations into sparse, potentially interpretable features.
10. Oct 14, 2026 · Research paper · 1 source
    Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability
    Mechanistic interpretability aims to decompose neural network computations into human-understandable features, yet progress is limited by the absence of ground-truth units to evaluate against.
11. Oct 14, 2026 · Research paper · 2 sources
    Sparse Autoencoders Are a Shared Coordinate System for Symbolic Matching
    Our method, SAE-IDF, encodes a symbol in its native context into a sparse code with a sparse autoencoder (SAE), re-weights each dimension by its rarity and matches across systems.
12. Oct 14, 2026 · Research paper · 1 source
    Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility
    Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, yet little work has tested whether this scale is necessary.
13. Oct 13, 2026 · Research paper · 1 source
    Readout Scaling Shapes Sparse Feature Recovery
    Readout scaling changes what a sparse autoencoder (SAE) learns, but the benefit of removing it depends on the data.
14. Oct 13, 2026 · Research paper · 1 source
    Evaluating the Causal Faithfulness of SHAP as a Candidate Attribution Method for SAE Features
    We apply SHAP-based attribution to SAE latent features and evaluate it against direct causal intervention, using sentiment classification on SST-2 with GPT-2 Small and a 32,768-feature layer-7 SAE as a testbed.
15. Oct 13, 2026 · Research paper · 1 source
    Confine to the Task, Anchor the Rest: Concept-Guided Retention Regularizer for Parameter-Efficient Fine-Tuning
    In this paper, we propose Concept-Guided Retention (CGR), a plug-and-play regularizer that constrains both for any PEFT base.
16. Oct 11, 2026 · Research paper · 1 source
    Small area learning
    We introduce small area learning (SAL), a learner-agnostic framework that formulates SAE as a multi-target semi-supervised learning problem and treats outcome prediction as a nuisance task.
17. Oct 11, 2026 · Research paper · 1 source
    What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions
    Sparse autoencoder (SAE) features are typically discovered from how their activations co-vary with behaviour, yet they are often used through interventions such as steering or ablation.
18. Oct 9, 2026 · Research paper · 1 source
    Cross-Modal Latent Influence: Patch-Level Training Sample Attribution for Vision-Language Models
    Recent latent-mediated influence methods provide feature-level attribution in language models, but lack the multimodal and spatial structure needed to ground influence in VLMs. We introduce MOSAIC (Multimodal Orthogonalized Sparse Autoencoders for Influence Characterization).
19. Oct 9, 2026 · Research paper · 1 source
    Language Models Have Interpretable Narrative Mechanisms
    We investigate whether language models contain internal components that contribute to representing narrativity, the degree to which a text exhibits narrative properties.
20. Oct 9, 2026 · Research paper · 1 source
    Sparse Dictionary Learning for Measure-Valued Data via Linearized Optimal Transport: Efficient Algorithms and Sample Complexity
    We study dictionary learning for probability measure-valued data by embedding measures into Hilbert spaces using features derived from optimal transport geometry.
21. Oct 8, 2026 · Research paper · 1 source
    RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routing
    Leveraging the SSH, we introduce RouterInterp, a method for interpreting expert routing that identifies Sparse Autoencoder features most predictive of routing decisions and produces unified natural language explanations.
22. Oct 8, 2026 · Research paper · 1 source
    Regularized Small Area Estimation with Graph Laplacian Benchmarking priors
    The resulting family includes a Benchmarking Prior that incorporates the benchmarking restrictions without additional regularization across areas, and Single and Multi-View Laplacian Benchmarking Priors that introduce regularization through graph Laplacians constructed from area similarities based on external covariate information.
23. Oct 7, 2026 · Research paper · 1 source
    Disentangling Linguistic and Paralinguistic Information with Routed Sparse Autoencoders
    Self-supervised speech encoders contain linguistic and paralinguistic information in a shared, entangled representation space.
24. Oct 7, 2026 · Research paper · 1 source
    Sparse Feature Policy Unlearning Mitigates State Hallucination in Vision-Language-Action Models
    Vision-Language-Action (VLA) models have shown strong generalization in robotic manipulation by leveraging rich representations from pretrained vision-language models.
25. Oct 6, 2026 · Research paper · 1 source
    AnyBottle: A Recipe to Only Keep the Concepts You Really Need
    We propose AnyBottle, a single recipe for building compact, task-specific CBMs. AnyBottle assumes only a frozen backbone and an unsupervised concept pool, such as a sparse autoencoder.
26. Oct 6, 2026 · Research paper · 1 source
    Illusory Pattern Perception Drives Spurious Inference in Large Language Models
    Illusory pattern perception is a well-documented human cognitive tendency to infer meaningful relationships in data that is actually random.

Often appears with