ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?

Key points

• Large language model (LLM) agents are increasingly used for vulnerability analysis, yet existing benchmarks provide limited coverage of exploitation that depends on interactions among network protocol components.1
• Tasks are primarily informed by published protocol-attack mechanisms.1
• Agents interact with running systems through scoped interfaces to achieve security-effect objectives without target source code.1
• The results show that agents can achieve verifiable exploitation effects in some cross-component environments, but success with limited black-box information remains low and varies across domains and attack categories.1

Sources (1)

• [1]ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 06:56 AM

  We introduce ProtoArena, a black-box, multi-component, dynamic benchmark with 80 cross-component vulnerability-exploitation environments spanning 11 protocol and application domains and 7 attack categories.

  Large language model (LLM) agents are increasingly used for vulnerability analysis, yet existing benchmarks provide limited coverage of exploitation that depends on interactions among network protocol components.

Extractive summary: sentences quoted from the sources.