Futarchy's fundamental flaw
32niplav
9dynomight
25RobinHanson
6dynomight
2RobinHanson
5dynomight
2RobinHanson
3Christopher King
2dynomight
13cousin_it
6dynomight
3cousin_it
3dynomight
2cousin_it
1dynomight
7So8res
5dynomight
6So8res
3dynomight
7So8res
6Dweomite
2[comment deleted]
2mako yass
11Ruby
11Baram Sosis
8dynomight
6Daniel C
6dynomight
3Daniel C
2dynomight
1Daniel C
4dynomight
3Daniel C
2dynomight
1Daniel C
2dynomight
1Daniel C
2dynomight
1Daniel C
3Mikhail Samin
3Matrice Jacobine
2rain8dome9
2Clay S
1_nuchi
1Bunthut
1Kyle Pena
1Austin Welch
1Vin
-1deepthoughtlife
2[comment deleted]
1[comment deleted]
New Comment
Previous discussion of the same (similar?) point:
- Futarchy implements evidential decision theory (Caspar Oesterheld, 2017)
- Conditional prediction markets are evidential, not causal (philh, 2024)
Futarchy also has some other issues, see e.g.
3The earliest I'm aware of is the 2015 post I linked at the end. (Though as far as I know, my impossibility theorem for alternate payoff strategies is novel.)
Do I correctly understand that you claim that under some plausible assumptions, the market will converge to P(A|do(B))? Can you state what those assumptions are? The challenge for me is that you go through a set of at least four possible sets of assumptions and give informal arguments for each. But I can't tell which of these sets of assumptions you believe is realistic, and under which of these you claim market prices will converge to p(A|do(B)). (Feel free to make simplifying assumptions like an infinite number of traders, no market fees, etc.)
Further, when you state that my result is wrong, that would seem to imply that no additional assumptions are needed. Yet all your arguments seem to rely on additional assumptions, which makes me question if my result really is wrong as stated, or rather that you prefer to add some additional assumptions.
Market estimates will converge to the most profitable P(X if A), the one that wins bets vs other versions. And that is the version you want to use when you make decisions.
Sorry to be persistent, but can you confirm that this means you do not claim markets in general converge to p(A|do(B))? That's my central claim, so when you state that I'm wrong, the obvious interpretation would be that that you believe that central claim is wrong. In your post, you don't identify what mistake supposedly exists, so I'd like to confirm if you're actually claiming to refute my central claim or if, rather, you're arguing that it doesn't matter.
You say that markets give evidential conditionals while decisions want causal conditionals. For this comment, I'm not taking a position on which conditional we want for decisions. I'm just saying that both trades and the decision advised should use the same conditional, but I'm not saying which one that is.
I'm a bit confused. What happens in a concrete example where CDT and EDT normally give a different answer?
For example, would a futarchy one-box (evidential decision theory) or two-box (casual decision theory)?
I'm a bit confused. Let's say we're deciding between actions X and Y. We set up a pair of conditional prediction markets to determine which would give us more utility. And we promise to follow the market's decision - nothing can influence our choice between X and Y, except the market's decision. In this situation, is there still a difference between conditional and causal probabilities?
1I think so. That's basically what I'm trying to argue in this section: https://www.lesswrong.com/posts/vqzarZEczxiFdLE39/futarchy-s-fundamental-flaw#Putting_markets_in_charge_doesn_t_work
(An alternative argument would be that by default P(A|B) != P(A|do(B)), so maybe the burden of proof should fall on whoever is claiming something special happens that makes them be the same in this case. It's possible that there are some assumptions under which this happens, but to the best of my knowledge, no one has specified what these assumptions would be, or made the argument that they're sufficient.)
Can you explain that section a bit? By your argument, it makes sense to pay at least 50 cents for the contract. But if that argument was correct, then everyone else in the market would also apply it, and the price of the contract would be at least 50 cents with certainty. That seems weird.
Sure, but could you help me understand exactly where the argument is failing for you? (E.g. with which of the three thought experiments?)
I should emphasize, I'm only claiming that the price will be at least 50 cents in the scenario where the coin is laser-scanned, so you know the market only activates when the coin has at least a 50% probability of landing heads. In the scenario where market activation is based on market prices, I only claim that you will pay more than 40 cents (probably less than 10 cents more), because the the fact that the market price closed above 50 cents still provides some extra info on top of your initial guess about the coin.
Yeah, I'm trying to think about the market scenario. Still can't tell if it works. If everyone in the market used the argument you describe, would the extra info actually be nonzero for everyone?
It's quite hard to argue against this in the abstract. My view would be that the burden of proof falls on the person who claims it works out. I've shown that in general people will bid at prices that don't reflect their true beliefs. It's possible there exists some set of assumptions where the market price converge to the true "market beliefs". But I'd like to see someone actually make that argument! It's very hard for me to prove that no such assumption+proof exist, because non-existence proofs are typically very hard. (See P=NP etc.)
That said, I do think the extra info would indeed almost always still be nonzero. People in general have different beliefs about the value of the coin. It's incredibly rare to start with a bunch of interrelated random variables and end up with something independent. If you were about to bet, and then I told you what the final market closing price would be, wouldn't you update your beliefs at least a little?
the trick is that the argument stops working for conditions that start to look like they might trigger. so the argument doesn't disrupt the idea that conditional prediction markets put the highest price on the best choice, but it does disrupt the idea that the pricings for unlikely conditions are counterfactually accurate.
for intuition, suppose there's a conditional prediction market for medical treatments for cancer. one of the treatments is "cut off the left leg." if certain scans and tests come back just the right way (1% chance likely) then cutting off the left leg is the best possible treatment, but otherwise, it's a dumb idea. if that condition is trading as if it's very unlikely to be a good idea, you can buy it up at very low risk -- most likely, the contract is canceled and you get your money back. but on the off-chance that the scans and tests come back in just the right way, you make a killing.
however, this incentive only exists insofar as "cut off the left leg" is not at risk of winning (before the tests and scans come back). if you thought the leg was going to be cut off simply because everyone else was buying the price up, and that it wouldn't actually heal the cancer, you'd sell off your shares.
this argument implies that you can't trust conditional prediction markets with cardinal ranking. however, afaict you'd need other arguments to imply that you can't trust their choice of "best action." such arguments probably exist, but this one isn't sufficient for it. (off the top of my head, i'd consider the case where option A is better for society and option B is better for some malicious actor, and the malicious actor is rich enough to convince the market to take option B. my intuition without thinking further is that this should 'obviously' work if the malicious actor is rich enough, in a fashion that's disanalogous to prediction markets in that it's not solved automatically with time (because the counterfactuals never settle and so the more-correct traders can't drain the rich-manipulator's money), but i haven't actually thought about it.)
the trick is that the argument stops working for conditions that start to look like they might trigger.
Can you give an argument for this claim? You're stating that there's an error in my argument, but you don't really engage with the argument or explain where exactly you think the error is.
For example, can you tell me what's incorrect in my example of two coins where you think one has a 60% probability and the other 59%, yet you'd want to pay more for a contract on the 59% coin? https://www.lesswrong.com/posts/vqzarZEczxiFdLE39/futarchy-s-fundamental-flaw#No__order_is_not_preserved (If you believe something is incorrect there.)
short version: the analogy between a conditional prediction market and the laser-scanner-simulation setup only holds for bids that don't push the contract into execution. (similarly: i agree that, in conditional prediction markets, you sometimes wish to pay more for a contract that is less valuable in counterfactual expectation; but again, this happens only insofar as your bids do not cause the relevant condition to become true.)
longer version:
suppose there's a coin that you're pretty sure is biased such that it comes up heads 40% of the time, and a contract that pays out $1 if the market decides to toss the coin and it comes up heads, and suppose any money you pay for the contract gets refunded if the market decides not to toss the coin. suppose the market will toss the coin if the contracts are selling for more than 50¢.
your argument (as i understand it) correctly points out that it's worth buying the contract from 40¢ to 45¢, because conditional on the market deciding to toss the coin, probably the market figured out that the coin actually isn't biased away from heads (e.g. via their laser-scanner and simulator). and so either your 45¢ gets refunded or the contract is worth more than 45¢, either way you don't lose (aside from the opportunity cost of money). but note that this argument depends critically on the step "the contract going above 50¢ is evidence that the market has determined that the coin is biased towards heads." but that argument only holds insofar as the people bidding the contracts from (say) 49¢ to 51¢ have actually worked out the coin's real bias (e.g., have actually run a laser-scanner or whatever).
intuition pump: suppose you bid the coin straight from 40¢ to 51¢ yourself, by accident, while still believing that the coin was very likely 40% likely to come up heads. the market closes in 5 minutes. what should you do? surely the answer is not "reason that, because the market price is above 50¢, somebody must have figured out that the coin is actually biased towards heads;" that'd be madness. sell.
more generally, nobody should bid a conditional branch into the top position unless they personally believe that it's worth it in counterfactual expectation. (or in other words: the conditional stops meaning "somebody else determined this was worth it" when you're the one pushing it over the edge; so when it comes to the person pushing the contract into the execution zone, from their perspective, the conditional matches the counterfactual.)
1Again, if there's a mistake, it would be helpful if you could explain exactly what that mistake is. You're sort of stating that the conclusion is mistaken and then giving a parallel argument for a different conclusion. It would be great (for multiple reasons) if you could explain exactly where my argument fails.
It might be helpful to focus on this example, which is pretty self-contained:
Suppose there’s a conditional prediction market for two coins. After a week of bidding, the markets will close, whichever coin had contracts trading for more money will be flipped and $1 paid to contract-holders for head. The other market is cancelled.
Suppose you’re sure that coin A, has a bias of 60%. If you flip it lots of times, 60% of the flips will be heads. But you’re convinced coin B, is a trick coin. You think there’s a 59% chance it always lands heads, and a 41% chance it always lands tails. You’re just not sure which.
We want you to pay more for a contract for coin A, since that’s the coin you think is more likely to be heads (60% vs 59%). But if you like money, you’ll pay more for a contract on coin B. You’ll do that because other people might figure out if it’s an always-heads coin or an always-tails coin. If it’s always heads, great, they’ll bid up the market, it will activate, and you’ll make money. If it’s always tails, they’ll bid down the market, and you’ll get your money back.
You’ll pay more for coin B contracts, even though you think coin A is better in expectation. Order is not preserved. Things do not work out.
Are you claiming that this is mistaken, or rather that this is correct but it's not a problem? (Of course, if this example is not central to what you see as a mistake, it could be the wrong thing to focus on.)
I've seen one argument which seems related to the one you're making and I do agree with. Namely, right before the market closes the final bidder has an incentive to bid their true beliefs, provided they know they will be the final bidder. I certainly accept that this is true. If you know the final closing price, then Y is no longer a random variable, and you're essentially just bidding in a non-conditional prediction market. I don't think this is completely reassuring on its own, though, because there's a great deal of tension with the whole idea of having a market equilibrium that reflects collective beliefs. I think you might be able to generalize this into some kind of an argument that as you get closer to closing, there's less randomness in Y and so you have more of an incentive to be honest. But this worries me because it would appear to lead to weird dynamics where people wait until the last second to bid. Of course, this might be a totally different direction from what you're thinking.
Are you claiming that this is mistaken, or rather that this is correct but it's not a problem?
mistaken.
But if you like money, you’ll pay more for a contract on coin B.
this is an invalid step. it's true in some cases but not others, depending on how the act of paying for a contract on coin B (with no additional knowledge of whether it's double-headed) affects the chance that the market tosses coin B.
you’ll pay more for a contract on coin B. You’ll do that because other people might figure out if it’s an always-heads coin or an always-tails coin. If it’s always heads, great, they’ll bid up the market, it will activate, and you’ll make money. If it’s always tails, they’ll bid down the market, and you’ll get your money back.
So8res seems to be arguing that this reasoning only holds if your own purchase decision can't affect the market (say, if you're making a private bet on the side and both you and your counter-party are sworn to Bayesian secrecy). If your own bet could possibly change which contract activates, then you need to worry that contract B activates because you bid more than your true belief on it, in which case you lose money in expectation.
(Easy proof: Assume all market participants have precisely the same knowledge as you, and all follow your logic; what happens?)
I think dynomight's reasoning doesn't quite hold even when your own bet is causally isolated, because:
- In order for you to pay more than $.59, you need to believe that the market is at least correlated with reality; that it's more likely to execute contract B if contract B actually is more valuable. (This is a pretty weak assumption, but still an important one.)
- In order for you to pay more than $.60 (not merely $.59 + epsilon), you not only need to believe that the market is correlated with reality, you need a quantitative belief that the correlation has at least a certain strength (enough to outweigh $.01). It's not enough for it to be theoretically possible that someone has better info than you; it needs to be plausible at a certain quantitative threshold of plausibility.
You can sort-of eliminate assumption #2 if you rework the example so that your true beliefs about A and B are essentially tied, but if they're essentially tied then it doesn't pragmatically matter if we get the order wrong. Assumption #2 places a quantitative bound on how wrong they can be based on how plausible it is that the market outperforms your own judgment.
At some point I'm gonna argue that this is a natural dutch book on CDT. (FDT wouldn't fall for this)
Curated. The idea of using Futarchy and prediction markets to make decision markets was among the earliest ideas I recall learning when I found the LessWrong/Rationality cluster in 2012 (and they continue to feature in dath ilani fiction). It's valuable then to have an explainer for fundamental challenges with prediction markets. I suggest looking at the comments and references, as there's some debate here, but overall I'm glad to have this key topic explored critically.
In your theorem, I don't see how you get that . Just because the conditional expectation of is the same doesn't mean the conditional expectation of is the same (e.g. you could have two different distributions over with the same expected value conditional on but different shapes, and then have depend non-linearly on , or something similar with ). It seems like you'd need some stronger assumptions on or whatever to get this to work. Or am I misunderstanding something?
(Your overall point seems right, though)
1Ack, I think you're right. I think I need to replace the assumption that 𝔼₁[Z | Y≥c] = 𝔼₂[Z | Y≥c] with the assumption that ℙ₁[(Y,Z)|Y≥c] = ℙ₂[(Y,Z)|Y≥c] which will guarantee the equality you're pointing out is true for all f. This seems totally fine since it's just a construction, but it definitely looks like an error to me. I'll fix that, thank you!
1Suppose you run a market where if you pay x and the final market price is y and z happens, then you get a payout of f(x,y,z) dollars. The payout function can be anything, subject only to the constraint that if the final market price is below some constant c, then bets are cancelled, i.e. f(x,y,z)=x for y < c.
But in futarchy the "threshold price" c wouldn't be constant, it would be the price of the market conditional on the scenario y<c.
IIUC the theorem is saying that you would be indifferent to whatever happens to Z if y<c, but that counterfactual would be estimated by another market (which estimates c) that activates when y<c and cancels when y>=c
I think that's right. (I guess technically it depends on what version of Futarchy you're trying to use. You could have a single market for a single coin that's flipped iff the final price is above some threshold.)
In general I agree this doesn't capture the complexity of how markets would get resolved. For example, the most common case would probably be that you have two markets for two coins and you resolve whichever one has a higher price. That doesn't fit into my assumptions.
I guess I was implicitly trying to argue that it doesn't work even with this extra simplifying assumption.
But I think you can also pretty easily generalize the proof? Suppose we change C to be a random variable (an arbitrary random variable, which I think can reflect pretty much any market design), and we change the payout function to be f(x,y,z,c) with the restriction that f(x,y,z,c)=x for y<c. Then I think the same proof strategy still works, just changing f(x,Y,Z) to f(x,Y,Z,C) everywhere?
I think that's right. (I guess technically it depends on what version of Futarchy you're trying to use. You could have a single market for a single coin that's flipped iff the final price is above some threshold.)[...] That doesn't fit into my assumptions.
Yep agreed.
But I think you can also pretty easily generalize the proof? [...] just changing f(x,Y,Z) to f(x,Y,Z,C) everywhere?
The proof definitely shows that within a single market (e.g. conditional on y>=c), you would be indifferent to Z given the opposite counterfactual (y<c), but that's okay because (in the two market for two coin case) we have two markets (estimating y and c) and each of the market would respond to each of the two possible counterfactuals (y>=c or y<c).
So although I would be indifferent between in the first market (conditional on y>=c) when they imply different distribution on Z conditional on y<c, I would not be indifferent between in the second market (conditional on y<c), and that difference would be reflected in c, which affects whether y>=c or y<c (& therefore which decision will be chosen).
Can you help me understand what you're claiming? Is it fair to think of your argument as supporting one of the following conclusions?
1. There is a function f with f(x,y,z,c)=x for y<c such that E[f]=E[z] always.
2. Rational agents would have some beliefs about the conditional distribution of c, and there is some function f for which that property is true once you add that assumption.
3. That property isn't necessary, some other (weaker) property is all that's needed.
I think I'm claiming 3, namely all we want from futarchy is for it to select the decision with the highest expected payout, and for that the property isn't necessary.
Ex: For the two coin two market case, the first market's price estimates the expected payout if we flip coin one (& similarly for the second market), & while neither market satisfies the property (E[f]=E[z] always), we would still select the decision that leads to the higher expected payout (as we select the higher price), and that's all that's needed
How do you feel about this example, which gives a setup where you have an incentive to bid more for a coin you think has a lower expected value?
Suppose there’s a conditional prediction market for two coins. After a week of bidding, the markets will close, whichever coin had contracts trading for more money will be flipped and $1 paid to contract-holders for head. The other market is cancelled.
Suppose you’re sure that coin A, has a bias of 60%. If you flip it lots of times, 60% of the flips will be heads. But you’re convinced coin B, is a trick coin. You think there’s a 59% chance it always lands heads, and a 41% chance it always lands tails. You’re just not sure which.
We want you to pay more for a contract for coin A, since that’s the coin you think is more likely to be heads (60% vs 59%). But if you like money, you’ll pay more for a contract on coin B. You’ll do that because other people might figure out if it’s an always-heads coin or an always-tails coin. If it’s always heads, great, they’ll bid up the market, it will activate, and you’ll make money. If it’s always tails, they’ll bid down the market, and you’ll get your money back.
You’ll pay more for coin B contracts, even though you think coin A is better in expectation. Order is not preserved. Things do not work out.
More generally, what's the argument that the market will always select the decision that leads to he higher expected payout?
We want you to pay more for a contract for coin A, since that’s the coin you think is more likely to be heads (60% vs 59%). But if you like money, you’ll pay more for a contract on coin B. You’ll do that because other people might figure out if it’s an always-heads coin or an always-tails coin. If it’s always heads, great, they’ll bid up the market, it will activate, and you’ll make money. If it’s always tails, they’ll bid down the market, and you’ll get your money back.
Let's call "Bidding on B, hoping that other people will figure out if B is an always-head or always-tails coin" strategy X, and call "Figure out if B is an always-head or always-tail myself & bid accordingly, or if I can't, bid on A because it's better in expectation" strategy Y.
If I believe that sufficient number of people in the market are using strategy Y, then it's beneficial for me to use strategy X, and insofar as my beliefs about the market are accurate, this is okay, because sufficient number of people using strategy Y means the market will actually figure out if B is always-head or always-tail, then bid accordingly. So the market selects the right decision, insofar as my beliefs about the market is correct (Note that I'm never incentivized to place a bid on B so large that it causes B to activate, since I don't actually know if B is always-head).
On the other hand, if I believe that the vast majority of people in the market are using strategy X instead of strategy Y, then it's no longer beneficial for me to use strategy X myself, I should instead use strategy Y because the market doesn't actually do the work of finding out if coin B is always-head for me. Other traders who have accurate beliefs about the market will switch to strategy Y as well, until there is a sufficient number of trader to push the market towards the right decision.
So insofar as people have accurate beliefs about the market, the market will end up selecting the right decision (either sufficient number of people use strategy Y, in which case it's robust for me to use strategy X, or not enough people are using strategy Y, in which case people are incentivized to switch to Y)
More generally, what's the argument that the market will always select the decision that leads to he higher expected payout?
"Always" might be too strong, but very informally:
Suppose that we have we have decision d1 d2, with outcome/payoff u & conditional market prices p1 (corresponds to d1) p2 (corresponds to d2)
if p1>E[u|d1], then traders are incentivized to sell & drive down p1. Similarly they will be incentivized to bid up p1 if p1<E[u|d1]. So p1 will tend toward E[u|d1]. We can argue similar for p2 tending towards E[u|d2]
Since we choose the decision with the higher price, and prices tend towards the expected payoff given that decision, the market end up choosing the decision that leads to the higher expected payoff.
Let's call "Bidding on B, hoping that other people will figure out if B is an always-head or always-tails coin" strategy X, and call "Figure out if B is an always-head or always-tail myself & bid accordingly, or if I can't, bid on A because it's better in expectation" strategy Y.
Regarding this, I'll note that my logic is not that different traders are following different strategies. I assume that all traders are rational agents and will maximize their expected return given their beliefs. My intended setup is that you believe coin A and coin B could have the biases stated, but you also believe that if you were to aggregate your beliefs with the beliefs of other people, the result would be more accurate than your beliefs alone.
I think this feeds into my objection to this proof:
Suppose that we have we have decision d1 d2, with outcome/payoff u & conditional market prices p1 (corresponds to d1) p2 (corresponds to d2)
if p1>E[u|d1], then traders are incentivized to sell & drive down p1. Similarly they will be incentivized to bid up p1 if p1<E[u|d1]. So p1 will tend toward E[u|d1]. We can argue similar for p2 tending towards E[u|d2]
Since we choose the decision with the higher price, and prices tend towards the expected payoff given that decision, the market end up choosing the decision that leads to the higher expected payoff.
My main objection to this logic is that there doesn't seem to be any reflection of the idea that different traders will have different beliefs. (It's possible that the market does give causal estimates with that assumption, but it's definitely not an assumption I'd be willing to make, since I think the central purpose of prediction markets is to aggregate diverse beliefs.) All my logic is based on a setup where different traders have different beliefs.
So I don't think the condition "p1>E[u|d1]" really makes sense? I think a given trader will drive down that market iff their estimate of the utility conditioned on that market activating is higher than p1, i.e. if p1>E_i[u|d1, market 1 activates]. I'm claiming that for trader i, E_i[u|d1, market 1 activates] != E_i[u|d1], basically because the event that market 1 activates contains extra information, and this makes it unlikely that the market will converge to E[u|d1].
My main objection to this logic is that there doesn't seem to be any reflection of the idea that different traders will have different beliefs.[...] All my logic is based on a setup where different traders have different beliefs.
Over time, traders who have more accurate beliefs (& act rationally according to those beliefs) will accumulate more money in expectation (& vice versa), so in the limit we can think of futarchy as aggregating the beliefs of different traders weighted by how accurate their beliefs were in the past
So I don't think the condition "p1>E[u|d1]" really makes sense? [...]and this makes it unlikely that the market will converge to E[u|d1].
If I pay p1 for a contract in market 1, my expected payoff is:
(since I get my money back if d2/market 2 is activated)
this is negative iff and positive iff
and if we commit to using futarchy to choose the decision, then is chosen iff market 1 activates, so E_i[u|d1, market 1 activates] should equal E_i[u|d1]
If I pay p1 for a contract in market 1, my expected payoff is:
(since I get my money back if d2/market 2 is activated)
this is negative iff and positive iff
This is incorrect. There are two errors here:
- The first expectation needs to be conditioned on the market activating. (That is not conditionally independent of u given d1 in general.)
- Different people have different beliefs, so the expectations are different for different traders. You can't write "E" without specifying for which trader.
I agree that if you assume u is conditionally independent of market activation given d1 and that all traders have the same beliefs then the result seems to hold. But those assumptions are basically always false.
The first expectation needs to be conditioned on the market activating. (That is not conditionally independent of u given d1 in general.)
If we commit to using futarchy to choose decision, then market 1 activating will have exactly the same truth conditions as executing d1, so "market activating and d1" would be the exact same thing as "d1" itself (commiting to use futarchy to choose decision means we assign 0 probability to "first market activating & execute d2" or "Second market activating & execute d1")
Different people have different beliefs, so the expectations are different for different traders. You can't write "E" without specifying for which trader.
Yes, we can replace with E_i, and then argue that traders with accurate beliefs will accumulate more money over time, making market estimates more accurate in the limit
Yes, we can replace with E_i, and then argue that traders with accurate beliefs will accumulate more money over time, making market estimates more accurate in the limit
There's a chicken-and-egg problem here. You're assuming that markets are causal (meaning traders that are better at estimating causal probabilities) and then using that assumption to prove that markets are causal.
There's a chicken-and-egg problem here[...] and then using that assumption to prove that markets are causal.
That argument was more about accomodating "different traders with different beliefs", but here's an independent argument for market being causal:
When I cause a particular effect/outcome, that means I mediate the influence between the cause of my action and the effect/outcome of my action, the cause of my action is conditionally independent of the effect of my action given me
Futarchy is a similar case: There may be many causes that influence market prices, which in turn determines the decision chosen, & market prices mediate the influence between the cause of market prices (e.g. different traders' beliefs) and the decision chosen. Any information can only influence what decision will be chosen through influencing the market prices. This seems like what it means for market to be causal (In a bayesnet, the decision chosen will literally only have market prices as the parent, assuming we commit to using futarchy to choose decisions).
I’ve discussed this with some people about a month ago. I don’t believe this is a real flaw. Prediction markets can be structured to explicitly implement CDT, and I argue they normally do that on their own.
This seems related to the 5-and-10 problem? Especially @Scott Garrabrant's version, considering logical induction is based on prediction markets.
The best use of prediction markets is for decision makers to prove their competence. Or as Lizka suggests to advise voting public.
i'm trying to understand the point about the vitamin d example. if a futarchy market is set up to predict whether "increasing vitamin d consumption" will "increase average lifespan," wouldn't participants who believe that wealth (or another confounder) is the actual causal factor, and not vitamin d itself, be incentivized to bet against the vitamin d policy leading to increased lifespan?
the market's incentive structure seems designed to filter for beliefs about the causal efficacy of the proposed intervention, not merely correlations. if people believe wealth is the cause, they wouldn't expect a vitamin d policy to succeed in raising lifespans, and would bet accordingly. it appears there might be a slight confusion between correlation in observational data and the causal impact of a direct intervention as assessed by a prediction market.
I commented on the post as well, but repeating here — I think denominating conditional markets in shares of contracts of the event happening (rather than in $) is more natural and resolves some confusions. i.e. denominate a contract that pays out if A conditional on B in shares of contracts that pay out $1 if B.
Shares of (coin comes up heads, conditional on coin being flipped) will then be traded in currency denominated in shares of (coin is flipped).
If everyone knows the coin will be flipped only if the laser scan comes up estimating the bias at > 0.5, then it's no surprise if the market is trading at > 0.5, and that really is everyone's true belief — namely their true belief of the probability of the coin coming up heads conditional on the coin being flipped. (Not their unconditional belief of the probability of the coin coming up heads!)
If instead everyone knows the coin is flipped only if the market is trading at > 0.5, then (probably?) most people's estimate of the coin's bias won't be meaningfully different depending on whether they condition on the coin being flipped or not — it's not strong evidence of the coin's bias — and so I expect the price to remain at whatever the market's prior of the bias was.
But those are two different situations, and I don't think the first one provides useful intuition for the second one. Setting up the right market is still a skill issue — in your Elon firing example, one would learn more from it by announcing ahead of time that the firing decision itself will be determined only by the results of the market. Then one doesn't have to worry about "well maybe their conditional probability is really due to some outside factor...". (Of course making such an announcement might cause both results to trade downward, because the market thinks that's a dumb way to make firing decisions. That's why it's a skill issue. If it seems like a dumb way to make firing decisions, maybe don't make such an announcement.)
Suppose b is the true bias of the coin (which the supercomputer will compute). Then your expected return in this game is
𝔼[max(b, 0.50)] = 0.50 + 𝔼[max(b-0.50, 0)]
No. That formula would imply that, if the coin is 30% for sure and you buy it for 0.3, you make 0.2 in expectation, which you don't, you make 0 regardless of what price you buy at.
Note that this kind of problem has also shown up in decision theory more generally. This is a good place to start. In particular, it seems like your problem can be fixed with epsilon exploration (if it doesn't do so automatically, as per Soares), both the EDT and CDT variant should work.
There are specific conditions under which conditional probability differs from causal intervention. Suppose we are comparing conditional probability P(Y|X=x) to intervention P(Y|do(X=x)). When there are no "backdoor paths" from Y to X - which loosely speaking, are indirect paths of influence - then these are equal.
Can we build simulations that capture real-world causal relationships and optimize via RL, similar to the AI Economist paper?
Is one of the methods of getting causal estimates that you're going to write about simply to give each possible decision a non-zero random chance of being selected? Here's what Hanson wrote about this approach:
Conditioning on random decisions should very effective, but seems expensive. However, it seems much less expensive to sometimes randomly not do a policy change that you were going to do, than to sometimes randomly do a policy change you were not going to do. So I suggest that a futarchy system for considering and adopting proposals randomly reject say 5% of the changes that it would otherwise have accepted. This should ensure good estimates conditional on not adopting proposals, leaving only the potential problem of a decision selection bias distorting estimates conditional on adopting some proposal.
https://open.substack.com/pub/overcomingbias/p/decision-selection-bias
I believe that the most important fundamental flaw for Futarchy isn't what is written here, but that the flaw this essay identifies is in fact sufficient to be described as an important fundamental flaw. I do not believe it is possible to patch out the most fundamental flaw.
My thought is that the most fundamental flaw is that you are asking for how people think the question will be resolved, and not the thing that is being referenced, so I think that prediction markets in general should be seen as just a (slightly unusual) type of opinion polling where you can theoretically win money by guessing other people's opinion's in the future. It's very meta. Prediction markets are very heavily selected for a specific type of person, so it is also an extremely biased poll. I do not believe that the self-selection is notably for accuracy. The chance to win money is likely to lead to notably more rigor than many other forms of polling (but also attracts gamblers).
Futarchy is just an unusual patch on top of prediction markets, and doesn't fundamentally change what a prediction market is. Like noted in the essay, that question is: what is the world like if this happens? And not: what effect does doing this have?
