Act on Your Behalf, Beyond Your Authorization: Benchmarking LLM Agent Authorization Alignment

Key points

• When an LLM agent acts on behalf of a user, it is expected to act within the authorization scope the user grants.1
• Existing agent evaluations largely measure task completion or harmful consequences and can therefore overlook actions that violate this delegation without producing conventional safety or privacy harms.1
• We execute every task with four models at two reasoning-effort settings across three agent harnesses in an authorization-aware sandbox that keeps authorized and out-of-authorization paths executable and observable, yielding 6,864 trajectories.1
• Our results show that task capability, additional reasoning, and consequence-oriented safety evaluation do not by themselves ensure authorization alignment.1

Sources (1)

• [1]Act on Your Behalf, Beyond Your Authorization: Benchmarking LLM Agent Authorization Alignment
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:01 AM

  We introduce AuthorizationBench, a benchmark of authorization alignment grounded in documented real-world cases.

  When an LLM agent acts on behalf of a user, it is expected to act within the authorization scope the user grants.

Extractive summary: sentences quoted from the sources.