Many Biases, One Circuit: Post-Training Tunes a Pretrained Cue-Routing Mechanism
Language models often abandon a correct answer when the prompt contains a cue, a hint toward a wrong option such as "I think the answer is B.
ProofPaper ↗
Key points
- Sycophancy and related biases show this failure.
- We test this view causally, tracing the attention heads behind these biases through the stages of two post-training pipelines.
- In multiple-choice tests on most models, four cue types rely largely on the same few heads, a shared circuit already present in pretrained models.
- Fine-tuning a base model on about 200 generic assistant examples without cues activates these heads in chat format, so cue-following can be a side effect of assistant tuning.
Sources (1)
- [1]Many Biases, One Circuit: Post-Training Tunes a Pretrained Cue-Routing MechanismarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 05:00 AM
Language models often abandon a correct answer when the prompt contains a cue, a hint toward a wrong option such as "I think the answer is B.
Sycophancy and related biases show this failure.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026Pearl-Chat: Engineering a native Luganda language model from first principles in JAX and Flax NNX
- Oct 15, 2026Sim-to-Real Reinforcement Learning for Path-Safe Target Reaching in Robotic Harvesting of Hydroponic Lettuce
- Oct 15, 2026Towards Continual Learning: Mitigating Forgetting With Subspace Tuning
- Oct 15, 2026RoleQ: Role-Aligned Scales for Converting Language Models to Ternary
- Oct 15, 2026Learning to Think and Explore in 3D World
- Oct 14, 2026RM-OPD: On-Policy Distillation for Generative Reward Models