Epistemic status: shitpost.

Suppose that you are selecting for utility. Naively, you might think that this means you are selecting for utility, but actually this is not the case.

Sometimes greenbrownness is the selection target

Military gear is sort of greenbrownish.

Military Camouflage - how camouflage works & the science behind it | UF PRO  Blog

This occurs because the designers of military gear are selecting for keeping soldiers alive, which benefits from camouflage, which in the common environments works best if it is greenbrownish. However, it fails at this in nongreenbrownish environments. Hence, keeping soldiers alive is not the selection target of military gear.

Sometimes disutility is the selection target

In the prisoner's dilemma, the highest-utility outcome is (cooperate, cooperate).

The pay-off matrix in the Prisoners' Dilemma game. The first entry... |  Download Scientific Diagram

However, utility maximizers will end up in (defect, defect), which is strictly lower utility than (cooperate, cooperate). Thus, utility maximization might have utility minimization as the selection target.

Sometimes the superficial appearance of utility is the selection target

Let's say that you see a website banner saying "You are the 1000000000th visitor to the site. Click to receive your award!". This looks like something people would say if they want to give you something, so you click the banner.

Electronic Arts on X: "@GameStop https://t.co/ELEQRcboMQ" / X
 

This does not end well for you, but you went into it selecting it because it looked like it would end well for you. 

Conclusion

Beware about reasoning about utility maximizers as maximizing utility. Utility maximizers may instead be maximizing many other things that are unrelated to utility, and not be maximizing utility.

New to LessWrong?

New Comment
1 comment, sorted by Click to highlight new comments since: Today at 3:57 PM

Note also that it is dangerous to speak of "the" selection target, as the selection target of utility maximizers may depend on the environment. For instance if military gear was designed for places that were e.g. neon pink, it would probably be designed to be neon pink instead of greenbrownish.