AK — LessWrong

LESSWRONG
LW

AK — LessWrong

In this example, he told you that you were not in one of the places you're not in (the Vulcan Desert). If he always does this, then the probability is 1/4; if you had been in the Vulcan Desert, he would have told you that you were not in one of the other three.

That can't be right -- if the probability of being in the Vulcan Mountain is 1/4 and the probability of being in the Vulcan Desert (per the guard) is 0, then the probability of being on Earth would have to be 3/4.

Replying toWhy Universal Comparability of Utility?

AK8y

Why Universal Comparability of Utility?

I'm not sure about the first case:

if you don't have a VNM utility function, you risk being mugged by wandering Bayesians

I don't see why this is true. While "VNM utility function => safe from wandering Bayesians", it's not clear to me that "no VNM utility function => vulnerable to wandering Bayesians." I think the vulnerability to wandering Bayesians comes from failing to satisfy Transitivity rather than failing to satisfy Completeness. I have not done the math on that.

But the general point, about approximation, I like. Utility functions in game theory (decision theory?) problems normally involve only a small space. I think completeness is an entirely... (read more)

Replying toWhy Universal Comparability of Utility?

AK8y

Why Universal Comparability of Utility?

Thanks for this response. On notation: I want world-states, $W_{i}$ , to be specific outcomes rather than random variables. As such, $U (W_{i})$ is a real number, and the expectation of a real number could only be defined as itself: $E [U (W_{i})] = U (W_{i})$ in all cases. I left aside all the discussion of 'lotteries' in the VNM Wikipedia article, though maybe I ought not have done so.

I think your first two bullet points are wrong. We can't reasonably interpret ~ as 'the agent's thinking doesn't terminate'. ~ refers to indifference between two options, so if $A > B > C$ and $P$ ~ $B$ , then $A > P > C$ . Equating... (read more)

Why Universal Comparability of Utility?

Apologies if this is answered elsewhere and I couldn't find it. In AI reading I come across an agent's utility function, $U$ , mapping world-states to real numbers.
The existence of $U$ is justified by the VNM-utility theorem. The first axiom required for VNM utility is 'Completeness' -- in the context of AI this means for every pair of world-states, $W_{i}$ and $W_{j}$ , the agent knows $W_{i} > W_{j}, W_{i} < W_{j},$ or $W_{i}$ ~ $W_{j}$ .

Completeness over world-states seems like a huge assumption. Every agent we make this assumption for must already have the tools to compare 'world where, all else equal, the only food is peach ice cream' v. 'world where, all... (read more)