Richard_Bruns

Ward Farnsworth (2007) The Legal Analyst

My main goal in teaching my introductory Economics class is to give students a good set of mental tools for understanding the world. This semester, I had a student who already had a surprisingly good understanding of game theory and questions of knowledge and proof. As we talked after class, he mentioned that he had learned these things from a book assigned for an introductory law class. After I asked about the book, he lent it to me.

From the minute I started reading 'The Legal Analyst', I saw that it was consistently excellent. About two-thirds of it was a readable, intuitive, high-quality summary of things I... (read more)

Here is a simple moral rule that should make an AI much less likely to harm the interests of humanity:

Never take any action that would reduce the number of bits required to describe the universe by more than X.

where X is some number smaller than the number of bits needed to describe an infant human's brain. For information-reductions smaller than X, the AI should get some disutility, but other considerations could override. This 'information-based morality' assigns moral weight to anything that makes the universe a more information-filled or complex place, and it does so without any need to program complex human morality into the thing. It is just information theory, which... (read 371 more words →)