Black-Box Red-Teaming of Multi-Agent Systems via Reinforcement Learning

Key points

• Large language model (LLM) agents are increasingly deployed in multi-agent systems (MAS) to accomplish complex tasks.1
• Prior black-box red-teaming attacks mainly focus on single agents, but we find that these methods are far less effective in MAS, where multiple sub-agents may not directly interact with the user.1
• Extensive experiments show that our method substantially improves attack success rates compared to existing black-box attacks, with transferability across different backend LLMs and MAS structures.1
• These results underscore the vulnerabilities of multi-agent systems and the importance of developing stronger defenses against adaptive black-box adversaries.1

Sources (1)

• [1]Black-Box Red-Teaming of Multi-Agent Systems via Reinforcement Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 10:32 PM

  Therefore, we introduce ReMAS, the first reinforcement learning-based red-teaming framework tailored to MASs, which fine-tunes attacker LLMs to generate effective adversarial prompts for system prompt extraction.

  Large language model (LLM) agents are increasingly deployed in multi-agent systems (MAS) to accomplish complex tasks.

Extractive summary: sentences quoted from the sources.