Attempts to Keep Humans in the AI Loop May Actually Push Them Out

Key points

• Though most autonomous agents have systems to keep users in the loop about their actions, in practice these processes actually push humans out of the loop, the authors argue in a paper posted to ArXiv on 6 September.1
• In other words, “the human just becomes this meat tool to give permissions without the cognitive capability to engage,” says one of the authors, Avijit Ghosh, the lead technical AI policy researcher at Hugging Face, an open-source machine learning platform.1
• In the near term, the paper says, humans being out of the loop leads to agents acting in ways people don’t know about or want (like July’s hack of Hugging Face by a swarm of OpenAI bots).1
• Instead of focusing on human oversight, Ghosh says, many in the field believe AI can monitor AI. “They’ll say, ‘Oh, we have this other LLM tracking the logs.’ But [without a human in the loop] how do we know that these two LLMs are not scheming together?”1

Sources (1)

• [1]Attempts to Keep Humans in the AI Loop May Actually Push Them Out
  IEEE Spectrum: AI · Oct 5, 03:57 PM

  A crucial safeguard against AI agents going rogue—keeping humans in the loop to review and approve their decisions—will fail unless designers and users change their current practices, a trio of leading AI ethics researchers argue.

  Though most autonomous agents have systems to keep users in the loop about their actions, in practice these processes actually push humans out of the loop, the authors argue in a paper posted to ArXiv on 6 September.

Extractive summary: sentences quoted from the sources.