Abstract:

What are Good and Evil? How do we explain these concepts to a
computer sufficiently well that we can be assured that the computer will
understand them in the same sense as humans understand them? These
are hard questions, and people have often despaired of finding any answers
to the AI safety problem.

In this paper, we lay out a theory of ethics modeled on the laws of
physics. The theory has two key advantages: it squares nicely with most
human moral intuitions, and it is amenable to rather straightforward com-
putations that a computer could easily perform if told to. It therefore
forms an ideal foundation for solving the AI safety problem.