Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning
We study affective state as an implicit state inferred from sparse contextual cues that remain relevant after topic shifts, and ask whether LLMs expose internal attention signals for routing such evidence at inference time.
ProofPaper ↗
Key points
- Large Language Models (LLMs) increasingly operate in multi-turn interactions that require maintaining latent conversational states, not only explicit facts.
- Prior attention interventions rely on labeled in-domain head profiles and externally specified target spans, limiting transfer across affective-evidence distributions.
- Across settings, SIAB improves prompt-matched baselines by up to +10.3 macro-F1 on ECC and +5.5 on GoEmotions.
- Combined with PASTA-style external supervision, it achieves up to +13.84 accuracy points and +12.51 macro-F1 points, while mechanistic analyses trace the gains to sparse, emotion-enriched heads and show that layer proximity affects additivity.
Sources (1)
- [1]Emotion Is Not External Knowledge: Internal Attention Signals for Affective State ReasoningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 06:29 AM
We study affective state as an implicit state inferred from sparse contextual cues that remain relevant after topic shifts, and ask whether LLMs expose internal attention signals for routing such evidence at inference time.
Large Language Models (LLMs) increasingly operate in multi-turn interactions that require maintaining latent conversational states, not only explicit facts.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026The Density of Thought: Predicting and Steering Reasoning in Large Language Models
- Oct 15, 2026Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
- Oct 15, 2026From Long Context to Selective Memory:A Survey of Memory-Centric Compression for Large Language Models
- Oct 15, 2026Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries
- Oct 14, 2026mistralai/Timestral-209M
- Oct 14, 2026Sparse Autoencoders Are a Shared Coordinate System for Symbolic Matching