Acceptable Misalignment

Key points

• This challenge is how one can develop such systems in an ethically sound way when there are many unknowns during design time about their manner of deployment, what they can and will learn, and how these systems will act on the basis of what they have learned.1
• One response to this challenge is to align, as far as possible, the activities and goals of autonomous robots with human values, raising the question of how to make this alignment possible, a question known as the value alignment problem.1
• We argue that the value alignment problem faces a number of challenges that show that full alignment is both theoretically and practically impossible, meaning that misalignment incidents will keep occurring.1
• We introduce the concept of acceptable misalignment, formulate normative requirements for alignment mechanisms aimed at bringing misalignment incidents to an acceptable level, and we conclude by presenting a new methodological framework that consists of alignment mechanisms that can meet these requirements.1

Sources (1)

• [1]Acceptable Misalignment
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 02:42 PM

  There is a specific challenge designers face in developing autonomous robots capable of open-ended learning.

  This challenge is how one can develop such systems in an ethically sound way when there are many unknowns during design time about their manner of deployment, what they can and will learn, and how these systems will act on the basis of what they have learned.

Extractive summary: sentences quoted from the sources.