Language Is a Security Surface for Agent Guards

Key points

• Step-level guards decide, before each tool call, whether an agent's next action is safe given the user's request and the trajectory so far.1
• Users write those requests in many languages, while guard prompts and the benchmarks on which purpose-built guards are trained and evaluated are English.1
• We ask whether a guard's verdict depends on the language of the request.1
• The direction of the damage tends to follow guard size: open guards of 12-30B parameters block more benign steps (up to +5.0 pp false positives), and every significant loss of recall is in a guard below 10B (up to -7.4 pp).1

Sources (1)

• [1]Language Is a Security Surface for Agent Guards
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 05:19 PM

  We build LinguaGuardBench from the 1,220 labelled steps of TS-Bench AgentDojo-Traj: we translate only the user instruction into German, French, Turkish, Swahili and Amharic, and keep every tool call, observation and label byte-identical, so any change in a verdict is attributable to language alone.

  Step-level guards decide, before each tool call, whether an agent's next action is safe given the user's request and the trajectory so far.

Extractive summary: sentences quoted from the sources.