Let's talk about a specific example: the Ultimatum Game. According to EY the rational strategy for the responder in the Ultimatum Game is to accept if the split is "fair" and otherwise reject in proportion to how unfair he thinks the split is. But the only reason to reject is to penalize the proposer for proposing an unfair split -- which certainly seems to be "doing something conditional on the other actor’s utility function disvaluing it". So why is the Ultimatum Game considered an "offer" and not a "threat"?

Yeah, but what does "purposefully minimize someone else’s utility function" mean? The source code just does stuff. What does it mean for it to be "on purpose"?

It all depends on what you mean by "sufficiently intelligent / coherent actors". For example, in this comment Eliezer says that it should mean actors that “respond to offers, not to threats”, but in 15 years no one has been able to cash out what this actually means, AFAIK.

Here's Joe Carlsmith making the second argument: https://joecarlsmith.com/2022/01/17/the-ignorance-of-normative-realism-bot

It is often said that: “The conclusions of deductive reasoning are certain, whereas those of inductive reasoning are probable”. I think this contrast is somewhat misleading and imprecise, as the certainty of deductive conclusions just means that they necessarily follow from the premises (they are implied by the premises), but the conclusion itself might still be probabilistic.

Example: “If I have a fever, there’s a 65% probability that I have the flu. I have a fever. Therefore, there’s a 65% probability that I have the flu.”

There's something off about this example. In deductive reasoning, if A implies B, then A and C together also imply B. But if A is "I have a fever" and C is "I have the flu" then A and C do not imply "there’s a 65% probability that I have the flu" (since actually there is a 100% chance).

I think what is going on here is that the initial statement "If I have a fever, there’s a 65% probability that I have the flu" is not actually an instance of material implication (in which case modus ponens would be applicable) but rather a ceteris paribus statement: "If I have a fever, then all else equal there’s a 65% probability that I have the flu." And then the "deductive reasoning" part would go "I have a fever. And I don't have any more information relevant to whether I have the flu than the fact that I have a fever. Therefore, there’s a 65% probability that I have the flu."

Depends on how dysfunctional the society is.

Basically both of these arguments will seem obvious if you fall into camp #2 here, and nonsensical if you fall into camp #1.

Memento is easily one of the best movies about “rationality as practiced by the individual” ever made. [...] When the “map” is a panoply of literal paper notes and photographs, and the “territory” is further removed from one’s lived experience than usual… it behooves one to take rationality, bias, motivated cognition, unquestioned assumptions, and information pretty damn seriously!

Wasn't the main character's attempt at "rationality as practiced by the individual" kind of quixotic though? I didn't get the impression that the moral of the story was "you should be like this guy". He would have been better off not trying any complicated systems and just trying to get help for his condition in a more standard way...

Let’s say my p(intelligent ancestor) is 0.1. Imagine I have a friend, Richard, who disagrees.

No wait, the order of these two things matters. Is P(intelligent ancestor|just my background information) = 0.1 or is P(intelligent ancestor|my background information + the fact that Richard disagrees) = 0.1? I agree that if the latter holds, conservation of expected evidence comes into play and gives the conclusion you assert. But the former doesn't imply the latter.