Accept or Repair: Preserving Passage-Supported Answers in Contrastive Decoding
Large language models often answer from knowledge learned during training, even when a provided passage contradicts that knowledge.
ProofPaper ↗
Key points
- To address this problem, contrastive decoding contrasts the predictions under the full and a modified prompt at every token, but it also alters correct answers that the passage supports.
- We show that checking answers against the passage removes this trade-off.
- Based on this observation, we propose Accept-or-Repair (AoR), a training-free procedure.
- AoR keeps a sampled answer if the passage supports it.
Sources (1)
- [1]Accept or Repair: Preserving Passage-Supported Answers in Contrastive DecodingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 01:32 PM
Large language models often answer from knowledge learned during training, even when a provided passage contradicts that knowledge.
To address this problem, contrastive decoding contrasts the predictions under the full and a modified prompt at every token, but it also alters correct answers that the passage supports.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 21, 2026ARCANA: A Benchmark for Abstraction and Analogical Reasoning
- Oct 21, 2026Target Regeneration Shrinks the Gains of Draft-Training Objectives in Speculative Decoding
- Oct 20, 2026BitMTP: When Multi-Token Prediction Meets Low-Bit Large Language Models
- Oct 20, 2026Tracing Recurrent and Emergent Hallucination Patterns in Decomposed Text-to-SQL Generation
- Oct 20, 2026When Can LLM Reasoning Reliably Reuse Intermediate Results?
- Oct 19, 2026Beyond Attack Success Rate: Cost-Aware Metric for Evaluating Iterative LLM Jailbreaks