You are viewing revision 1.7.0, last edited by William_Quixote

Ethical injunctions are rules not to do something even when you believe it's the right thing to do. (That is, you refrain "even when your brain has computed it's the right thing to do", but this will just seem like "the right thing to do".)

For example, you shouldn't rob banks even if you plan to give the money to a good cause.

This is to protect you from your own cleverness (especially taking bad black swan bets), and the Corrupted hardware you're running on.

Related to the Metaethics sequence.


Power corrupts is well known folk wisdom. This post gives an evo-psych explanation. Corrupt behavior provides a fitness advantage, but signaling corruption makes it hard to get power. The cleanest way to not signal corruption is to honestly believe that one will not be corrupt. Thus the fittest strategy is to couple an honest desire to do good with a tendency to find the common abuses  of power pleasurable. 

This post is not cross listed as a part of the listed main sequences. 

"The end does not justify the means" is just consequentialist reasoning at one meta-level up.  If a human starts thinking on the object level that the end justifies the means, this has awful consequences given our untrustworthy brains; therefore a human shouldn't think this way.  But it is all still ultimately consequentialism.  It's just reflective consequentialism, for beings who know that their moment-by-moment decisions are made by untrusted hardware.

This post is not cross listed as a part of the listed main sequences. 

See also