Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence
Web search allows LLMs to access up-to-date information, but it remains unclear why they sometimes produce outdated answers even when relevant evidence is available.
ProofPaper ↗
Key points
- To study this problem, we construct temporal facts whose values changed after the models' training cutoffs.
- We evaluate one proprietary model under deployed web search and six open-weight model configurations under controlled retrieval.
- We find that models often answer from search snippets without opening the source pages, which is associated with higher rates of outdated answers and unfaithful citations.
- We hope these findings will help advance research on the reliable use of retrieved evidence in search-augmented LLMs.
Sources (1)
- [1]Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available EvidencearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 11:48 AM
Web search allows LLMs to access up-to-date information, but it remains unclear why they sometimes produce outdated answers even when relevant evidence is available.
To study this problem, we construct temporal facts whose values changed after the models' training cutoffs.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026[AINews] Modal’s $1B ARR, $16B Series D makes it NYC’s newest AI Infra Decacorn, 3x usage and 3x valuation in 5 months, 15x in 1 yr
- Oct 15, 2026Beyond Retrieval Failure: Understanding Why LLMs Miss Evidence in Multi-Hop RAG
- Oct 14, 2026Generate presentations with Amazon Bedrock AgentCore
- Oct 14, 2026Web search for a 1.7B model running fully on a phone, no search API and no backend (open source)
- Oct 13, 2026Online Tokenization with Optimal Delay in Linear Time
- Oct 13, 2026Plan or Retrieve First? A Scale Study of Tool Discovery Architectures