Towards Safe Autonomy for Computer Use

Key points

• The safety of LLM-based agents currently rests on alignment training and safety classifiers, neither of which can rule out that a sophisticated indirect prompt injection (IPI) goes through and causes significant harm.1
• Recent work therefore builds agents with information-flow guarantees that rely on structured tool interfaces to interact with the environment.1
• Computer Use Agents (CUAs) act visually on the computer screen, which does not provide a boundary between trusted first-party interfaces and untrusted, potentially attacker-authored content.1
• This allows the agent to autonomously handle tasks involving untrusted information.1

Sources (1)

• [1]Towards Safe Autonomy for Computer Use
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 03:03 PM

  We introduce SA-CUA, a dual-LLM framework that generates a reliable trust boundary for CUAs by deriving conservative deterministic rules from the accessibility tree for unmasking trusted regions from the privileged LLM's screenshot, keeping trusted and untrusted content separated.

  The safety of LLM-based agents currently rests on alignment training and safety classifiers, neither of which can rule out that a sophisticated indirect prompt injection (IPI) goes through and causes significant harm.

Extractive summary: sentences quoted from the sources.