Honeyed Words, Hidden Daggers: Benchmarking and Training LLMs to See Through Mock Politeness
We present ZhenhuanBench, a 1,592-item benchmark built from expert commentary on Empresses in the Palace, covering speaker intent, emotion-language mismatch, and per-listener uptake.
ProofPaper ↗
Key points
- The Chinese idiom "kou mi fu jian" ("honeyed words, hidden daggers") names speech that is polite in form but hostile in function.
- Text-only evaluation covers 34 LLMs. GPT-6 Astra leads the leaderboard at 73.48% overall, while its listener-reaction score is 58.37%.
- The annotations also supply supervised traces and string-verifiable training rewards.
- Without target-task training, CPR outperforms Base, SFT, and GRPO across six Chinese and English pragmatic and sarcasm evaluations.
Sources (1)
- [1]Honeyed Words, Hidden Daggers: Benchmarking and Training LLMs to See Through Mock PolitenessarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 09:19 AM
We present ZhenhuanBench, a 1,592-item benchmark built from expert commentary on Empresses in the Palace, covering speaker intent, emotion-language mismatch, and per-listener uptake.
The Chinese idiom "kou mi fu jian" ("honeyed words, hidden daggers") names speech that is polite in form but hostile in function.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 18, 2026TIRE: Top-Down Formula Instantiation with Reliable Execution for Physics Problem Synthesis
- Oct 17, 2026[AINews] High Taste Testers Ep. 1
- Oct 17, 2026EdgeKernelBench: Unified LLM-Driven Kernel Generation for Edge Backends
- Oct 16, 2026Prefix-Tuning for Arbitrary Output Sequences on Pretrained Transformers
- Oct 16, 2026Perceive, Solve, Reason: Progressive Capability Formation in Time Series-LLMs
- Oct 16, 2026axolotl-ai-cloud/axolotl v0.21.0