ToolHACE: A Dataset for Detecting Hallucinations in Tool-Based LLM Responses
To address this gap, we introduce ToolHACE, a span-level benchmark for hallucination detection in tool-augmented LLM responses, covering five fine-grained hallucination categories.
ProofPaper ↗
Key points
- Tool-augmented language models ground their responses in external tool outputs, improving reliability beyond parametric knowledge alone.
- Existing hallucination benchmarks for tool-augmented dialogue primarily focus on the tool-calling stage, evaluating whether the correct tool and arguments are selected, while overlooking errors that arise during post-tool response generation.
- Through extensive experiments with modern hallucination detectors, we show that post-tool-call hallucinations are widespread and remain challenging to detect.
- Our results further demonstrate that effective detection requires training on specialized tool-augmented corpora, while transfer from tool-agnostic hallucination datasets performs poorly in this setting.
Sources (1)
- [1]ToolHACE: A Dataset for Detecting Hallucinations in Tool-Based LLM ResponsesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 05:15 PM
To address this gap, we introduce ToolHACE, a span-level benchmark for hallucination detection in tool-augmented LLM responses, covering five fine-grained hallucination categories.
Tool-augmented language models ground their responses in external tool outputs, improving reliability beyond parametric knowledge alone.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026unslothai/unsloth v0.1.906-beta: Browser Use + Qwen-Image-2.1-Turbo
- Oct 11, 2026RIVO: Lightweight Adaptive Rerouting for Deep Research Agents
- Oct 11, 2026Apertus 1.5 Release Notes
- Oct 10, 2026FleetPilot: Fine-Grained Steering Empowers Real-World LLM Personalization
- Oct 10, 2026vllm-sr/d3
- Oct 10, 2026SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usability