Assessing the Empirical Evidence for Loss of Control from Agentic General-Purpose AI

Key points

• The EU's Code of Practice for General-Purpose AI lists loss of control as a systemic risk that must be assessed, yet no structured assessment of the quality of the empirical evidence for this risk exists.1
• We evaluate 19 recent empirical studies across eight properties identified from threat models (such as strategic deception, situational awareness, and autonomous replication), assessing the quality of evidence across three dimensions: construct validity, content validity, and external validity.1
• Our central finding is that the current body of evidence supports a judgment of loss of control as "weakly plausible.1
• Therefore, our findings highlight the need for more rigorous evaluations of agentic general-purpose AI systems.1

Sources (1)

• [1]Assessing the Empirical Evidence for Loss of Control from Agentic General-Purpose AI
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 01:03 PM

  Theoretical literature warns that advanced generalist agentic systems might pursue instrumental goals that are misaligned with human preferences, thus posing risks of irreversible loss of human control.

  The EU's Code of Practice for General-Purpose AI lists loss of control as a systemic risk that must be assessed, yet no structured assessment of the quality of the empirical evidence for this risk exists.

Extractive summary: sentences quoted from the sources.