## LESSWRONGLW

See my response here

You want to consider the utility of the terrorists, at the appropriate level of detail.

Huh? Yes it will. You mean "you will still find it undesirable and or hard for you to understand".

What are the units for expected utility? How do you measure them? Can you graph my utility function?

I can look at behaviors of people and say that on this day Joe bought 5 apples and 4 oranges, on this day he bought 2 kiwis, 2 apples and no oranges etc...but that data doesn't reliably forecast expected utility for oranges. There are so many exogenous variables that the data is reliably unreliable.

I have yet to see a researcher give a practical empirical formula mapping the utility of a person or group. I argue it is because it is impossible (currently), thus trying to do so doesn't make sense in practice. I have however, as demonstrated in the link previously, seen formulas which imply weighted preference set's. Those aren't any more useful or descriptive than saying that Joe prefers apples to oranges.

# 8

Recently I argued that the economist's utility function and the ethicist's utility function are not the same.  The nutshell argument is that they are created for different purposes - one is an attempt to describe the actions we actually take and the other is an attempt to summarize our true values (i.e., what we should do).  I just ran across a somewhat older post over at Black Belt Bayesian arguing this very point.  Excerpt:

Economics (of the neoclassical kind) models consumers and other economic actors as such utility maximizers... Utility is not something you can experience. It’s just a mathematical construct used to describe the optimization structure in your behavior...

Consequentialist ethics says an act is right if its consequences are good. Moral behavior here amounts to being a utility maximizer. What’s “utility”? It’s whatever a moral agent is supposed to strive toward. Bentham’s original utilitarianism said utility was pleasure minus pain; nowadays any consequentalist theory tends to be called “utilitarian” if it says you should maximize some measure of welfare, summed over all individuals... Take note: not all utility maximizers are utilitarians.

There’s no necessary connection between these two kinds of utility other than that they use the same math. It’s possible to make up a utilitarian theory where ethical utility is the sum of everyone’s economic utility (calibrated somehow), but this is just one of many possibilities. Anyone trying to reason about one kind of utility through the other is on shaky ground.