Do LMs Know What Variables Do? Interpreting Code Representations via Variable Roles

Key points

• Variable roles describe recurring behaviors such as accumulation, traversal, and retaining the best candidate encountered.1
• We introduce static annotation rules for all eleven roles and a corpus of 126,926 natural-code files across five languages.1
• Across five model-SAE systems, residual probes outperform individual SAE features and tested lexical baselines, with modest losses after neutral target renaming.1
• Combining SAE features closes much of the single-feature gap, but probes on SAE reconstructions are less sensitive to role-changing edits reviewed by LLMs. High paired ranking accuracy coexists with frequent false positives at fixed thresholds.1

Sources (1)

• [1]Do LMs Know What Variables Do? Interpreting Code Representations via Variable Roles
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 06:46 PM

  We use Sajaniemi's independently defined taxonomy to study which role distinctions can be recovered from large language models, how sparse autoencoders (SAEs) retain those distinctions, and whether associated directions support control of generation.

  Variable roles describe recurring behaviors such as accumulation, traversal, and retaining the best candidate encountered.

Extractive summary: sentences quoted from the sources.