Beyond Factual Accuracy: A Benchmark for Evaluating Perspective Alignment in LLMs

Key points

• Large Language Models often generate factually correct responses that fail to maintain user-requested perspectives, blending multiple perspectives and forcing users to filter through unwanted information---a perspective control failure that factual accuracy metrics cannot detect.1
• Experiments reveal that state-of-the-art models successfully include target content but fail to exclude non-requested perspectives.1
• Providing documents through RAG improves factual grounding but does not resolve perspective drift, as models prioritize comprehensive coverage over strict user constraints.1

Sources (1)

• [1]Beyond Factual Accuracy: A Benchmark for Evaluating Perspective Alignment in LLMs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 07:49 AM

  We introduce PerspectiveBench, a benchmark quantifying perspective alignment by constructing scenarios where multiple valid reasoning paths coexist, and propose the Perspective Alignment Score (PAS) to measure adherence to specified perspectives.

  Large Language Models often generate factually correct responses that fail to maintain user-requested perspectives, blending multiple perspectives and forcing users to filter through unwanted information---a perspective control failure that factual accuracy metrics cannot detect.

Extractive summary: sentences quoted from the sources.