Query-Adaptive Neuron Steering for LLMs
Based on these findings, we propose Q-NeuS, a query-adaptive neuron steering framework that dynamically determines which neurons to modulate and by how much for each input.
ProofPaper ↗
Key points
- Effectively eliciting the capabilities of large language models (LLMs) at inference time remains an important challenge.
- This raises a natural question: Can internal steering in LLMs be made query-adaptive?
- In this work, we investigate whether query-induced internal activations can guide adaptive steering in LLMs. Our pilot studies reveal subject- and layer-dependent neuron activation patterns and show that amplifying subject-relevant neurons can improve model performance.
- To learn effective intervention strategies, we introduce Neu-GRPO, which adapts group-relative optimization to train the Query-Adaptive Gate using the relative performance of multiple neuron intervention configurations.
Sources (1)
- [1]Query-Adaptive Neuron Steering for LLMsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 03:30 AM
Based on these findings, we propose Q-NeuS, a query-adaptive neuron steering framework that dynamically determines which neurons to modulate and by how much for each input.
Effectively eliciting the capabilities of large language models (LLMs) at inference time remains an important challenge.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 10, 2026UltraBERT: Beyond the Limit of BERT Pre-Training
- Oct 10, 2026ProgAgent: Learning Tool Orchestration through Programmatic Training
- Oct 10, 2026Vela: User Behavior as Supervision Sparks LLM Intelligence
- Oct 9, 2026ASSET: Adaptive Speech Segmentation via Reinforcement Learning
- Oct 9, 2026The Thinking Leakage in Reinforcement Learning for LLMs
- Oct 9, 2026Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding