Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents
Tool-using agents read untrusted external content, which exposes them to indirect prompt injection (IPI), where hidden instructions hijack the agent's tool calls.
ProofPaper ↗
Key points
- Benchmarks measure IPI security by the attack success rate (ASR), which treats every failed attack as a successful defense.
- Our Attack-Task Outcome Matrix (ATOM) further shows that in 21.7% of AgentDojo episodes, the agent stayed safe but did not complete the user's task.
- A low ASR therefore mostly reflects agents silently ignoring injections, not refusing them.
- We release the labeled corpus, judge prompt, and analysis code.
Sources (1)
- [1]Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 10:26 AM
Tool-using agents read untrusted external content, which exposes them to indirect prompt injection (IPI), where hidden instructions hijack the agent's tool calls.
Benchmarks measure IPI security by the attack success rate (ASR), which treats every failed attack as a successful defense.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026anujasid/blender-ai-mcp-addon: AI assistant and MCP server for Blender: visual chat, scene tools, multi-agent workflows. Works with ChatGPT/GPT, Claude, Gemini, Ollama, Op
- Oct 15, 2026Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint
- Oct 15, 2026The Model Knows Better: Defending Medical Multi-Agent Systems Against Prompt Injection
- Oct 15, 2026NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
- Oct 15, 2026GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations
- Oct 14, 2026INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems