In his talk AI Alignment: Why It’s Hard, and Where to Start Eliezer Yudkowsky gives "an interesting experiment in cognitive psychology" about a hospital administrator who has to decide whether to spend $1M on a sick child or "general hospital salaries, upkeep, administration, and so on". Yudkowsky mentions that
"A lot of the subjects in the cognitive psychology experiment became very angry and wanted to punish the administrator for even thinking about the question."
Yudkowsky thinks this is ironic, because they "are prohibiting a priori efficiently using money to save lives".
I couldn't find the experiment in question and Yudkowsky gives no reference to it. But let's assume that these people wanted to... (read 451 more words →)