Do Emergent Misalignment Models Share a Circuit?
Fine-tuning language models on narrowly defined harmful tasks can induce emergent misalignment (EM) across unrelated domains.
ProofPaper ↗
Key points
- While prior work has identified shared misalignment-associated representations across independently fine-tuned models, it remains unclear whether these models rely on common computational components.
- Using direction-based activation tracing, we identify three attention-head locations shared among the five highest-ranked heads of all three organisms.
- Cross-organism activation transfer further shows that the shared heads exhibit partially interchangeable effects on fixed-response preference.
- These findings provide evidence that representational convergence in EM extends to shared, causally relevant attention-head components within a common model architecture.
Sources (1)
- [1]Do Emergent Misalignment Models Share a Circuit?arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 08:16 PM
Fine-tuning language models on narrowly defined harmful tasks can induce emergent misalignment (EM) across unrelated domains.
While prior work has identified shared misalignment-associated representations across independently fine-tuned models, it remains unclear whether these models rely on common computational components.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026NVIDIA/TensorRT-LLM v1.4.0rc0
- Oct 14, 2026Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training
- Oct 14, 2026The Design Space of Tri-Modal Masked Diffusion Models
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
- Oct 13, 2026unslothai/unsloth v0.1.907-beta: Large Performance Gains + More Browser Use
- Oct 13, 2026inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs