AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety

Key points

• Large language model (LLM) agents accomplish complex tasks through multi-turn interactions with external tools.1
• Adversarial co-evolution has emerged as an effective approach to strengthening LLM safety, but existing work primarily focuses on language-level safety or indirect prompt injection in web tasks.1
• We further introduce learnability-guided task weighting, which uses within-task defender-reward variability to adjust each task's contribution to attacker and defender updates.1
• Extensive experiments on agent safety and general capability benchmarks demonstrate that our method improves adversarial robustness while maintaining general agent capabilities.1

Sources (2)

• [1]AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 02:13 AM

  In this paper, we introduce AgentSafeEvo, a co-evolutionary reinforcement learning framework for agent safety built on an attacker-defender game spanning diverse tasks and attack types.

  Large language model (LLM) agents accomplish complex tasks through multi-turn interactions with external tools.

• [2]AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety
  Hugging Face Daily Papers · Oct 14, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.