Where and Why Search-Augmented LLMs Fail When Reasoning over Rapidly Evolving Knowledge
To address this gap, we introduce TREK, a suite of three components: TREKTRACE is a plug-and-play framework that attributes failures to specific stages of retrieval and reasoning.
ProofPaper ↗
Key points
- As the world changes, new information is published while older records remain online.
- This complicates reasoning for large language models (LLMs), which must retrieve evidence and resolve conflicts to determine which information remains valid.
- We apply it to TREKBENCH-WILD, a meta-benchmark of real-world questions that require complex reasoning over rapidly evolving knowledge.
- To study this bottleneck, we build TREKBENCH-CTRL, a controlled benchmark with distractors spanning temporality, relevance, and trustworthiness, and show that benchmarks dominated by outdated distractors substantially underestimate task difficulty.
Sources (1)
- [1]Where and Why Search-Augmented LLMs Fail When Reasoning over Rapidly Evolving KnowledgearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 03:09 PM
To address this gap, we introduce TREK, a suite of three components: TREKTRACE is a plug-and-play framework that attributes failures to specific stages of retrieval and reasoning.
As the world changes, new information is published while older records remain online.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 17, 2026Otaku — an LLM frontend for creative writing
- Oct 17, 2026Blackbox Performance Analysis of LLM-based Network Control
- Oct 16, 2026ECHO: Replacing the Compliance Proxy with Response Harmfulness in Optimization-Based Jailbreak Attack
- Oct 16, 2026Uncertainty-Aware Budget Allocation for Efficient LLM Tree Search
- Oct 16, 2026Language Models Learn to Improve Language Models
- Oct 15, 2026Discovering state-of-the-art quantum circuits with large language models