You are viewing revision 1.9.0, last edited by William_Quixote

Ethical injunctions are rules not to do something even when you believe it's the right thing to do. (That is, you refrain "even when your brain has computed it's the right thing to do", but this will just seem like "the right thing to do".)

For example, you shouldn't rob banks even if you plan to give the money to a good cause.

This is to protect you from your own cleverness (especially taking bad black swan bets), and the Corrupted hardware you're running on.

Related to the Metaethics sequence.


Power corrupts is well known folk wisdom. This post gives an evo-psych explanation. Corrupt behavior provides a fitness advantage, but signaling corruption makes it hard to get power. The cleanest way to not signal corruption is to honestly believe that one will not be corrupt. Thus the fittest strategy is to couple an honest desire to do good with a tendency to find the common abuses  of power pleasurable. 

This post is not cross listed as a part of the listed main sequences. 

"The end does not justify the means" is just consequentialist reasoning at one meta-level up.  If a human starts thinking on the object level that the end justifies the means, this has awful consequences given our untrustworthy brains; therefore a human shouldn't think this way.  But it is all still ultimately consequentialism.  It's just reflective consequentialism, for beings who know that their moment-by-moment decisions are made by untrusted hardware.

This post is not cross listed as a part of the listed main sequences. 

Most lies, in order to stand against rigorous investigation, would require additional lies about supporting facts. Since people do not know all aspects of all disciplines, the web of supporting lies will eventually entail making a claim that is self evidently false to someone with expert knowledge the liar does not possess. Only a god could lie to an AI.

Part of the Against Rationalization subsequence of How To Actually Change Your Mind

A more personal / reflective post in which Eliezer looks back and observes that his ethically motivated truthfulness has led to better outcomes than he would have achieved by lying. He proposes several reasons for this including that honesty makes it harder to sweep problems away forcing him to deal with them. 

This post is not cross listed as a part of the listed main sequences. 

See also