Evaluating the Causal Faithfulness of SHAP as a Candidate Attribution Method for SAE Features

Key points

• Sparse autoencoders decompose language model representations into interpretable features, but feature interpretation in this literature remains predominantly correlational.1
• Attribution methods offer more direct estimates of feature influence, yet their faithfulness to actual causal effects has not been systematically evaluated in the SAE setting.1
• We compare SHAP to Integrated Gradients, Gradient$\times$Input, and probe-weight baselines across linear and MLP probe arms, scoring each against feature ablation and activation steering measured through the language model's own logit difference.1
• Second, under known-ground-truth minimal pairs, attribution recovers inserted sentiment signal well above chance (AUROC $0.792$) but fails to localize deletions (AUROC $0.520$), suggesting that these methods detect the presence of a feature's influence more reliably than its absence.1

Sources (1)

• [1]Evaluating the Causal Faithfulness of SHAP as a Candidate Attribution Method for SAE Features
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 07:54 AM

  We apply SHAP-based attribution to SAE latent features and evaluate it against direct causal intervention, using sentiment classification on SST-2 with GPT-2 Small and a 32,768-feature layer-7 SAE as a testbed.

  Sparse autoencoders decompose language model representations into interpretable features, but feature interpretation in this literature remains predominantly correlational.

Extractive summary: sentences quoted from the sources.