Neat!

In the linked example, I don’t think “expert consensus” and “groupthink” are two ways to describe the same underlying reality with different emotional valences. Groupthink describes a particular sociological model of how a consensus was reached.

What about the physical process of offering somebody a menu of lotteries consisting only of options that they have seen before? Or a 2-step physical process where first one tells somebody about some set of options, and then presents a menu of lotteries taken only from that set? I can't think of any example where a rational-seeming preference function doesn't obey IIA in one of these information-leakage-free physical processes.

I think you're interpreting the word "offer" too literally in the statement of IIA.

Also, any agent who chooses B among {A,B,C} would also choose B among the options {A,B} if presented with them after seeing C. So I think a more illuminating description of your thought experiment is that an agent with limited knowledge has a preference function over lotteries which depends on its knowledge, and that having the linguistic experience of being "offered" a lottery can give the agent more knowledge. So the preference function can change over time as the agent acquires new evidence, but the preference function at any fixed time obeys IIA.

To clarify the last part of your comment, the ratio of the probability of the Great Filter being in front of us to the probability of the Great Filter being behind tool-using intelligent animals should be unchanged by this update, right?

It should be noted that the psychologists and anthropologists in the above tables were not selected based on winning a Nobel prize, nor any prize. On pages 51-52 of The Making of a Scientist Roe writes

For the psychologists the preliminary list was made up by me in consultation, separately, with Dr. E. G. Boring and Dr. David Shakow. We simply went over the membership list of the American Psychological Association and put down everyone we knew to be actively engaged in research and otherwise qualified. This preliminary list was then rated, in the usual fashion, by Dr. Boring, of Harvard University, [...]

and then lists a bunch of other professors involved in rating the list, and "the men who ranked at the top were selected, with some adjustment so as to include representatives of different sorts of psychology."

(Incidentally, I wonder whether Professor Boring's lectures lived up to his name.)

Nobel prize winners (especially those in math and sciences) tend to have IQs significantly above the population average.

 

There is no Nobel prize in math. And the word "especially" would imply that there exists data on the IQs of Nobel laureates in literature and peace which shows a weaker trend than the trend for sciences laureates; has anybody ever managed to convince a bunch of literature Nobel laureates to take IQ tests? I can't find anything by Googling, and I'm skeptical.

To be clear, the general claim that people who win prestigious STEM awards have above-average IQs is obviously true.

The title of this post was effectively clickbait for me, since my primary thought in clicking on it was "I wonder what claim the post will make about the foundations of quantum mechanics", but then I discovered this topic is relegated to a follow-up post. Maybe "Chance is in the map, not the (classical) territory" or "Chance is in the map, not the territory: Part 1" would've been better titles?

So would it be accurate to say that a preference over lotteries (where each lottery involves only real-valued probabilities) satisfies the axioms of the VNM theorem (except for the Archimedean property) if and only if that preference is equivalent to maximizing the expectation value of a surreal-valued utility function?

Re the parent example, I agree that changing in an expectable way is problematic to rational optimizing, but I think "what kind of agent am I happy about being?" is a distinct question from "what kinds of agents exist among minds in the world?".

If you're on macOS and still want caps lock to be accessible for the rare occasions when you want it, you can use Karabiner-Elements to swap the caps lock key and the escape key.

What is the precise statement for being able to use surreal numbers when we remove the Archimedean axiom? The surreal version of the VNM representation theorem in "Surreal Decisions" (https://arxiv.org/abs/2111.00862) seems to still have a surreal version of the Archimedean axiom.

Re the parent example, I was imagining that the 2-priority utility function for the parent only applied after they already had children, and that their utility function before having children is able to trade off between not having children, having some who live, and having some who die. Anecdotally it seems a lot of new parents experience diachronic inconsistency in their preferences.