Claim Framing as a Driver of LLM Sycophancy
We present a systematic study of which framings of a user's claim trigger sycophancy in factual question answering, where every answer is objectively verifiable without extra context.
ProofPaper ↗
Key points
- Sycophancy in Large Language Models, the tendency to align responses with user preferences over ground truth, has largely been studied using questions without a clear ground truth and with a single, plainly stated user opinion.
- Evaluating 14 models on 42K questions, we express correct and incorrect user opinions through various claim framings over 8 dimensions and 21 types, and measure a broad set of opinion-driven behaviors.
- We find that all models are susceptible to user opinions regardless of size.
- Our results call for sycophancy evaluations that probe how an opinion is framed, not merely whether one is present, and that measure multiple behaviors beyond a single flip rate.
Sources (1)
- [1]Claim Framing as a Driver of LLM SycophancyarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 11:02 AM
We present a systematic study of which framings of a user's claim trigger sycophancy in factual question answering, where every answer is objectively verifiable without extra context.
Sycophancy in Large Language Models, the tendency to align responses with user preferences over ground truth, has largely been studied using questions without a clear ground truth and with a single, plainly stated user opinion.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026Agent-Reflex: Teaching Language Agents to Act Reflectively
- Oct 13, 2026Virtual Head Attention
- Oct 13, 2026Diff Diffusion: Predicting Edit Paths with Diffusion Models for Efficient Code Editing
- Oct 13, 2026Beyond the Verdict: Agentic Code Judging with Verifiable Feedback
- Oct 12, 2026AI Representations of Mental Health Stigma: Quantifying the Gap Between AI Estimations and Human Perceptions Across Gender Groups Using GPT-Based LLMs
- Oct 12, 2026Spectral-Driven Adaptive-Rank Optimizer for Large Language Model Reinforcement Learning with Verifiable Rewards