CWE-bench: Refocusing Agentic Security from Exploitation to Defense
Cybersecurity evaluations of AI agents have focused largely on offense, yet a working exploit does not show that an agent can make software safe.
ProofPaper ↗
Key points
- An attacker needs to exploit only one vulnerability; a defender must find and fix all of them.
- We introduce CWE-bench, a private benchmark of 120 held-out repository audit-and-repair tasks spanning 73 CWE types, eight languages, and all OWASP Top 10 2025 categories.
- We further conduct analyses showing that (i) agents with the same pass@1 differ widely in retry gains and cost, and (ii) no single agent is best across languages or vulnerability categories, and difficulty varies far more by vulnerability category than by language.
- As the offensive capabilities of agents improve, CWE-bench provides a broad and rigorous measure of whether their defensive capabilities keep pace.
Sources (1)
- [1]CWE-bench: Refocusing Agentic Security from Exploitation to DefensearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 04:00 PM
Cybersecurity evaluations of AI agents have focused largely on offense, yet a working exploit does not show that an agent can make software safe.
An attacker needs to exploit only one vulnerability; a defender must find and fix all of them.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026Alibaba Qwen Introduces QwenGyre: Elastic Reinforcement Learning RL for XLong-Horizon Agents
- Oct 12, 2026AI models may be capable of self-replicating without access to a datacenter
- Oct 12, 2026Strengthening our defenses against distillation attacks
- Oct 11, 2026RayenMlayeh/jevcrawl: MCP server and CLI that crawls websites for AI agents: finds the answer page 2x more often than Crawl4AI, 10x cheaper per answer than lettin
- Oct 11, 2026OrcaRouter Releases OrcaCyber Zero 1.5 Cybersecurity Model With 1M Context
- Oct 10, 2026Anthropic is cutting off its internal evaluations from the internet