No, the whole premise of the face-reading scenario is that the agent can tell that his face is being read, and that's why he pays the money. If the agent can't tell whether his face is being read, then his correct action (under FDT) is to pay the money if and only if (probability of being read) times (utility of returning to civilization) is greater than (utility of the money). Now, if this condition holds but in fact the driver can't read faces, then FDT does pay the $50, but this is just because it got unlucky, and we shouldn't hold that against it.
In your new dilemma, FDT does not say to pay the $50. It only says to pay when the driver's decision of whether or not to take you to the city depends on what you are planning to do when you get to the city. Which isn't true in your setup, since you assume the driver can't read faces.
a random letter contains about 7.8 (bits of information)
This is wrong, a random letter contains log(26)/log(2) = 4.7 bits of information.
I have tinnitus every time I think about the question of whether I have tinnitus. So do I have tinnitus all the time, or only the times when I notice?
I was confused at first what you meant by "1 is true" because when you copied the post from your blog you didn't copy the numbering of the claims. You should probably fix that.
The number 99 isn’t unique—this works with any payoff between 30 and 100.
Actually, it only works with payoffs below 99.3 -- this is the payoff you get by setting the dial to 30 every round while everyone else sets their dials to 100, so any Nash equilibrium must beat that. This was mentioned in jessicata's original post.
Incidentally, this feature prevents the example from being a subgame perfect Nash equilibrium -- once someone defects by setting the dial to 30, there's no incentive to "punish" them for it, and any attempt to create such an incentive via a "punish non-punishers" rule would run into the trouble that punishment is only effective up to the 99.3 limit.
It's part of the "frontpage comment guidelines" that show up every time you make a comment. They don't appear on GreaterWrong though, which is why I guess you can't see them...
I explained the problem with the votes-per-dollar formula in my first post. 45% of the vote / $1 >> 55% of the vote / $2, so it is not worth it for a candidate to spend money even if they can buy 10% of the vote for $1 (which is absurdly unrealistically high).
When I said maybe a formula would help, I meant a formula to explain what you mean by "coefficient" or "effective exchange rate". The formula "votes / dollars spent" doesn't have a coefficient in it.
...If one candidate gets 200 votes and spends 200 dollars, and candidate 2 gets 201 votes and spen
I don't think the data dependency is a serious problem, all we need is a very loose estimate. I don't know what you mean by a "spending barrier" or by "effective exchange rate", and I still don't know what coefficient you are talking about. Maybe it would help if you wrote down some formulas to explain what you mean.
I don't understand what you mean; multiplying the numerator by a coefficient wouldn't change the analysis. I think if you wanted to have a formula that was somewhat sensitive to campaign spending but didn't rule out campaign spending completely as a strategy, Votes/(10X+Y) might work, where Y is the amount spent of campaign spending, and X is an estimate of average campaign spending. (The factor of 10 is because campaign spending just isn't that large a factor to how many votes you get in absolute terms; it's easy to get maybe 45% of the vote with no campaign spending at all, just by having (D) or (R) in front of your name.)
The result of this will be that no one will spend more than the $1 minimum. It's just not worth it. So your proposal is basically equivalent to illegalizing campaign spending.
I wonder whether this one is true (and can be easily proved): For a normal form game G and actions ai for a player i, removing a set of actions a−i from the game yields a game G− in which the Nash equilibria are worse on average for i (or alternatively the pareto-best/pareto-worst Nash equilibrium is worse for G− than for G).
It's false: consider the normal form game
(0,0) (2,1)
(1,1) (3,0)
For the first player the first option is dominated by the second, but once the second player knows the first player is going to choose the second option, he's moti...
Not eating meat is not a Pascal's mugging because there is a solid theoretical argument for why the expected value is positive even if the payoff distribution is somewhat unbalanced: if a large number of people decide not to eat meat, then this will necessarily have the effect of shifting production, for supply to meet demand. Since you have no way of knowing where you are in that large ensemble, the expected value of you not eating meat is equal to the size of the effect divided by the number of people in the ensemble, which is presumably what we would ex...
A proof you don’t understand does not obligate you to believe anything; it is Bayesian evidence like anything else. If an alien sends a 1GB Coq file Riemann.v, running it on your computer does not obligate you to believe that the Riemann hypothesis is true. If you’re ever in that situation, do not let anyone tell you that Coq is so awesome that you don’t roll to disbelieve. 1GB of plaintext is too much, you’ll get exhausted before you understand anything. Do not ask the LLM to summarize the proof.
I'm not sure what you are trying to say here. Even with 1...
Thus, for example, intransitivity requires giving up on an especially plausible Stochastic Dominance principle, namely: if, for every outcome o and probability of that outcome p in Lottery A, Lottery B gives a better outcome with at least p probability, then Lottery B is better (this is very similar to “If Lottery B is better than Lottery A no matter what happens, choose Lottery B” – except it doesn’t care about what outcomes get paired with heads, and which with tails).
This principle is phrased incorrectly. Taken literally, it would imply that the mixe...
It seems like "equally probable MWI microstates" is doing a lot of work here. If we have some way of determining how probable a microstate is, then we are already assuming the Born probabilities. So it doesn't work as a method of deriving them.
That quote seems nonsensical. What do the Born probabilities have to do with a counting argument, or with the dimension of Hilbert space? A qubit lives in a two-dimensional space, so a dimension argument would seem to suggest that the probabilities of the qubit being 0 or 1 must both be 50%, and yet in reality the Born probabilities say they can be anything from 0% to 100%. What am I missing?
-"“Percentage of marriages that end in divorce” is an underspecified concept. There is only “percentage of marriages that end in divorce after n years”. "
The concept is perfectly well specified, just take n to be e.g. 75. But of course, it can only be measured for cohorts that are at least that old. Still, I would have assumed it possible to do some extrapolation to estimate what the value will be for younger cohorts (e.g. the NYT article you linked to says "About 60 percent of all marriages that eventually end in divorce do so within the first 10 years", ...
-"If you have heard that “40% of marriages end in divorce” or some similar figure, you are probably misinterpreting the divorce-to-marriage ratio. "
Really? So what is the right number then? A cursory Google search shows 40-50% is a commonly repeated figure for "percentage of marriages that end in divorce", are you really claiming that all of those webpages are misinterpreting the divorce-to-marriage ratio? What is the basis for such a claim? It does not appear to be in the Wikipedia article, which says nothing about the percentage of marriages that end in divorce.
To explain my disagree-vote: I think such a system would necessarily create a strong bias against downvotes/disagree-votes, since most people would just not downvote rather than making a justifying comment. "Beware trivial inconveniences"
The infinite autoresponse example seems like it would be solved in practice by rational ignorance: after some sufficiently small number of autoresponses (say 5) people would not want to explicitly reason about the policy implications of the specific number of autoresponses they saw, so "5+ autoresponses" would be a single category for decisionmaking purposes. In that case the induction argument fails and "both people go to the place specified in the message as long as they observe 5+ autoresponses" is a Nash equilibrium.
Of course, this assumes people haven...
-"this is just the lie algebra, and is why elements of it are always invertible."
First of all, how did we move from talking about numbers to talking about Lie algebras? What is the Lie group here? The only way I can make sense of your statement is if you are considering the case of a Lie subgroup of GL(n,R) for some n, and letting 1 denote the identity matrix (rather than the number 1) [1]. But then...
Shouldn't the Lie algebra be the monad of 0, rather than the monad of 1? Because usually Lie algebras are defined in terms of being equipped with two operati...
-"On any finite dim space we have a canon inner product by taking the positive definite one."
What? A finite dimensional space has more than one positive definite inner product (well, unless it is zero-dimensional), this choice is certainly not canonical. For example in R^2 any ellipse centered at the origin corresponds to a positive definite inner product.
I know this is not your main point, but the Millenium Problems are not an instance of a way for an AGI to quickly get money. From the CMI website:
-"Before CMI will consider a proposed solution, all three of the following conditions must be satisfied: (i) the proposed solution must be published in a Qualifying Outlet (see §6), and (ii) at least two years must have passed since publication, and (iii) the proposed solution must have received general acceptance in the global mathematics community"
-"The more time passes, the more evidence we get that (4) is false from computational tests and also the more we should doubt any proof (3) as its complexity grows. Therefore, the logical probability of (1) and (2) is growing over time."
The fact that the methods we use to check proofs are not perfectly reliable (and I think you greatly overstate the importance of this consideration, computer proof checkers are very reliable) does not at all imply that the probability that a proof exists is decreasing over time. You need to distinguish between the fact of a...
Just to be clear, Bertrand's postulate is actually a theorem (i.e. a known result), not a postulate/hypothesis. It is called a "postulate" for historical reasons.
-"Bayesians reason about truth this way because they are designed to treat knowledge as justified, true belief, reflecting a popular theory in analytic philosophy of what knowledge is."
Hasn't that theory been discredited (by Gettier)? I don't think it is popular anymore.
Let's limit our attention to the three hypotheses (a) there is no correlation between names and occupations, (b) the Pelham paper is right that Dennises are about 1% more likely to go into dentistry, and (c) the effect is much larger, e.g. Dennises are 100% more likely to go into dentistry. Then Bayes' theorem says observing a Dennis in dentistry increases the odds ratio P(b)/P(a) by a factor of 1% and the odds ratio P(c)/P(a) by a factor of 100%. You say you consider (a) and (b) to each have prior probability of 50%, which presumably means (c) has negligi...
I think quantum mechanics and the MWI are a red herring here. The real question is whether you can compute the probability of counterfactual events like WWII not happening -- and as Viliam says, the answer to that question is that it depends on choosing a starting point in the past to diverge from. Of course, once you choose such a point, actually estimating the probability of WWII not happening is an exercise in historical reasoning, not quantum physics.
-"One way to put this is that they aren’t made true by our desires, goals, standards, values, beliefs, and so on."
OK, I am a moral realist under this formulation.
-"Rather, they are true in a way more like how claims about e.g., the mass of an object are true."
I guess it depends on what you mean by "in a way more like". Moral claims are pretty fundamentally different from physical claims, I don't see how to get around that -- one way to put it would be that the notions of right and wrong are not inductive generalizations over observed phenomena -- another w...
-"In doing so, they aren’t appeal to their own values, or anyone else’s values, but to facts about what’s morally right or wrong that are true without reference to, and in a way that doesn’t depend on, any particular evaluative standpoint."
OK, so now it sounds like I am not a moral realist! I definitely think that by making a moral claim you are appealing to other people's values, since other people's values is the only thing that could possibly cause them to accept your moral claim. However, the moral claim is still of the form "X is true regardless of whether it is consistent with anyone's values".
-"I think the central question would be: Do you think that there are facts about what people morally should or shouldn’t do, or what’s morally good or bad, that are true independent of people’s goals, standards, or values? If yes, that’s moral realism. If not, that’s moral antirealism."
I certainly don't believe that the truth of moral facts is dependent on people's goals, standards, or values; the qualifier I would give is that our beliefs about moral facts are the same thing (tautologically) as our moral standards. So I guess I am a moral realist? Or mayb...
(Disclaimer: didn't read the post, it is too long and I doubted it would engage with my views.)
I'm not sure how popular moral anti-realism actually is here. For example, Eliezer's position was technically moral realist, though his metaethics was kind of strange.
I'm not sure whether to classify myself as a moral realist or anti-realist. Regarding your litmus test "it's wrong to torture babies for fun" I find myself saying that it's true in a sense, but in a different sense than we normally use the word "true". How important this difference is depends on whe...
You seem to be claiming that it is possible for mathematical truths such as 2+2=5 to be other than what they are; I can agree with this on an epistemological level (since we don't know all mathematical truths) but on on ontological level, no: mathematical truths are necessary truths. This is the conventional view though I'm not really sure how to argue it to a skeptic: but if you don't see why 2+2=4 is a necessary truth then I claim you don't truly comprehend why 2+2=4.
Mathematical facts are facts about well-defined what-if scenarios. We evolved to be able to consider such scenarios because they often bear a resemblance to what happens to us. So there is an explanation for how our beliefs about mathematics could become correlated with mathematical truth, even though this explanation is not causal. However, it is not entirely obvious how to tell a similar story about moral truths -- why did we evolve to be able to perceive moral facts, if indeed we did?
-"For example, I could imagine laws requiring anyone scraping the internet to ensure that they are not collecting data from people who have denied consent to have their data scraped."
In practice this is already the case, anyone who doesn't want their data scraped can put up a robots.txt file saying so, and I imagine big companies like OpenAI respect robots.txt. I guess there could be advantages in making it a legal rule but I don't think it matters too much.
You seem to have misunderstood the problem statement [1]. If you commit to doing "FDT, except that if the predictor makes a mistake and there’s a bomb in the Left, take Right instead", then you will almost surely have to pay $100 (since the predictor predicts that you will take Right), whereas if you commit to using pure FDT, then you will almost surely have to pay nothing (with a small chance of death). There really is no "strategy that, if the agent commits to it before the predictor makes her prediction, does better than FDT".
[1] Which is fair enough, a...
-"Charging a toll for a bridge you didn’t build is not okay; that’s pure extraction."
This is probably just a nitpick, but as worded this doesn't take into account the scenario where the builder of the bridge sells the rights to charge a toll to another party, who can then legitimately charge the toll even though they didn't build the bridge.
Yes they do. For simplicity suppose there are only two hosts, and suppose host A precommits to not putting money host B's box, while host B makes no precommitments about how much money he will put in host A's box. Then the human's optimal strategy is "pick host A's box with probability 1 - x epsilon, where x is the amount of money in host A's box". This incentivizes host B to maximize the amount in host A's box (resulting in payoff ~101 for the human), but it would have been better for him if he had precommitted to do the same as A, since then by symmetry his box would have been picked half the time instead of 101 epsilon of the time.
Couldn't you equally argue that they will do their best not to be smallest by not putting any money in all their opponent's boxes? After all, "second-fullest" is the same as "third-emptiest".
Why would precommitting to pick the second-fullest box give an incentive for predictors to put money in everyone else’s boxes?
If the hosts move first logically, then TDT will lead to the same outcomes as CDT, since it's in each host's interest to precommit to incentivising the human to pick their own box -- once the host has precommitted to doing this, the incentive works regardless of what decision theory the human uses. In math terms, if x is the choice of which box to incentivize (with "incentivize your own box" being interpreted as "don't place any money in any of the other boxes"), the human gets to choose a box f(x) on the basis of x, and the host gets to choose x=g(f) on t...
You seem to be assuming the human moves first in logical time, before the superintelligent hosts. You also seem to be assuming that the superintelligent hosts are using CDT (if they use FDT, then by symmetry considerations all of their possible actions have equal payoff, so what they do is arbitrary). Any particular reason for these assumptions?
Where do the numbers $152 and $275 come from? I would have thought they should be $100 and $200, respectively.
In the 5 box problem, why doesn't FDT force all of the incentives into box 1, thus getting $400?
-"The main question is: In the counter-factual scenario in which TDT recommends action X to agent A , what does would another agent B do?"
This is actually not the main issue. If you fix an algorithm X for agent A to use, then the question "what would agent B do if he is using TDT and knows that agent A is using algorithm X?" has a well-defined answer, say f(X). The question "what would agent A do if she knows that whatever algorithm X she uses, agent B will use counter-algorithm f(X)" then also has a well-defined answer, say Z. So you could define "the res...
Can't this be answered by an appeal to the fact that the initial state of the universe is supposed to be low-entropy? The wavefunction corresponding to one of the worlds, run back in time to the start of the universe, would have higher entropy than the wavefunction corresponding to all of them together, so it's not as good a candidate for the starting wavefunction of the universe.