Raise your hand if you (yes you, the person reading this) will submit to 50 years of torture in order to avert "least bad" dust speck momentarily finding its way into the eyes of an unimaginably large number of people.
Why was it not written "I, Eliezer Yudkowsky, should choose to submit to 50 years of torture in place of a googolplex people getting dust specks in their eyes"?
Why restrict yourself to the comforting distance of omniscience?
Did Miyamoto Musashi ever exhort the reader to ask his sword what he should want? Why is this not a case of using a tool as an end in and of itself rather than as a means to achieve a desired end?
Are you irrational if your something to protect is yourself...from torture?
Has anyone ever addressed whether or not this applies to the AGI Utility Monster whose experiential capacity would presumably exceed the ~7 billion humans who should rationally subserve Its interests (whatever they may be)?
"Oh my gosh! 'The Sun goes around the Earth' is true for Hunga Huntergatherer, but for Amara Astronomer, 'The Sun goes around the Earth' is false! There is no fixed truth!" The deconstruction of this sophomoric nitwittery is left as an exercise to the reader.
An apt way to put it. That this worthless dimestore philosophy so often underlies contemporary contemplative discourse by relatively intelligent people never ceases to bewilder and sadden me. (see example below)
...a team of neurologists investigated a 40Hz electrical rhythm...
For the sake of the blook; neuroscientists, not neurologists. Words can be wrong.
I selected to cooperate.
If I'd thought the financial incentive to defect was greater, I may have been tempted to do so... ...but isn't it interesting that even a modest material reward didn't have the same effect as the incentive to lie about IQ?