Decision markets seem to have tricky problems with manipulation, see this paper for example. And even prediction markets, which are simpler, might have problems as well. I'd like to understand this better, is there a simple scheme that avoids the problems?
Here is the reason this is a bad idea in as short a form as possible.
Fundamentally the question you are asking is a regression question. Given present measurements about the world [x], what is the predicted future state of the world [y].
The actual real world problem - the amount of information in [x], and the relative determinism of our reality in respect to [x], limits how accurate a prediction can be even with a perfect algorithm trained on an infinite number of prior examples of [x].
Essentially the problem you are trying to solve is a restatement of most machine learning problems. In most machine learning problems, the problem is in this form, and if you want to be methodical, you try out a large number of possible algorithms and tuning parameters to solve the problem.
At the end of the day, you do the following:
1. Feed the problem (you need many examples of past data) into an engine that will try to find the best fit model
2. Evaluate the outcome
If a good model exists, use it's predictions to inform your decisions.
Either present methods can find a good model, or they cannot. Not every problem is solvable, and usually this is because it isn't possible. We could have a prediction market on the next number a hardware RNG will spit out, but obviously no one can do better than chance.
At the end of the day the problem I have with your prompt is you posit that someone exists that can do better than just paying for the best tools (or professional actuarial analysis) you can afford. (even if the tool is free you pay with your time)
The exception to this is when there is hidden information. For example, if there were a prediction market bet on "there will be an assassination attempt on [VIP name] in the next week". The problem here is the only individuals able to do better than chance are conspiring to commit the crime, and buying shares on the market would risk both alerting the security on the target, and revealing their real world identities and culpability.
Or "Apple will develop a car". Same idea - the people who actually know are almost all direct employees of apple and buying shares on the market is just a form of being paid to leak information.
The way you described the chess/marriage/etc market, it's a bit vulnerable. Imagine there is a move that appears to be a very strong one, but with a small possibility of a devastating countermove that is costly for market participants to analyze. There is an incentive to bet on it - if the countermove exists, hopefully somebody will discover it, heavily bet against the move, and cause the price to drop enough that it is not taken, and the bets are refunded. If no countermove exists, the bet is a good one, and is profitable. But if nobody bothers to check for the countermove, and it exists, everybody (those who bet on the move, and the decision makers who made the move) are in trouble, but it could still be the case that no bettors have enough incentive to check for countermove (if it exists, they do not derive any benefit from the significant mispricing of the move, as you just refund the bets).
Epistemic status: This seems sufficiently basic that it seems like someone would already have developed a general theory, but I haven’t managed to find anything. If you know of anything like this please let me know. This post copied over from here.
Say you want to make some really good choices – you want your business to succeed, your marriage to flourish, and to beat your long-term rival in a chess game. Problem is, you’re not very good at making your own decisions.
The obvious thing is to ask some experts for help. But there’s two problems with this: First, who even are the relevant experts? In chess this is simple enough – just call up the top ten or so players by ELO rating – but who will you call for your business and marriage? MBAs and marriage counsellors claim to have relevant expertise, but there’s no good rigorous data to check how good they are. Second, what do we do when the experts disagree? And how do we get them to help us in the first place?
The classic solution for this is to set up a betting market. The good experts will gain influence, money, and confidence proportionate to how good they actually are, and the Efficient Market Hypothesis implies this will give us the best possible decisions (assuming there’s enough money in the market to make it worth the competent experts’ time). This solves the last two problems, but “just set up a betting market” is doing a lot of work on what we actually need to do here.
The classic example people give of a betting market is the stock market, but in practice they’re pretty different things. In the stock market, investors bet on how well publicly traded companies will do[1], and make money if they’re right[2][3]. In theory the thing that backs this all up is that companies have the ability to give dividends when they make profits, and that if you buy enough of a company you get the power to control it, but in practice this is all a Keynesian beauty contest. The values of betting markets, on the other hand, are backed by the fact that they pay out directly if you’re right.
But this distinction doesn’t really matter – there’s other things, like commodities, where trading pretty closely resembles direct bets on a belief. The bigger issue with the equivalence is that there’s only about 3500 publicly-traded stocks in the entire US market. For comparison, there’s about 10^40 possible chess positions, which is far more unmanageable, if you were to just do the simple thing of trying to evaluate how good the various positions are. Marriage and business decisions are even more multibranched.
Here’s one solution for chess: At each turn, set up a betting market on which move you’re most likely to win with. Once the market closes, you make the move that had the best odds. Everyone who bet on a different move gets their money back. Bets on your move are kept in the pot, and get their winnings (from the people who bet for the other player’s betting market) if you end up winning the game. This technically works – the bettors’ incentives are aligned with having you win[4], and there’s always a manageable number of options – but there’s a lot of friction. It takes a lot of moves for a bet to pay off, and it’s pretty risky (even if you’re completely sure what the best move in one position is, there’s about forty other moves that could ruin it for you). The time lag is particularly bad for the other examples – you can finish a chess game reasonably fast if you set a ten minute time limit on the betting market for each move, but it takes much longer to see if your business or marriage will succeed.
One possible solution to this is to set up a secondary market (which none of the people in the betting market can bet on) reflecting the general state of your success – Just a market on how likely you are to win the chess game, for chess, or just the stock price, for your business – and have the primary market pay out in proportion to how much your stock went up or down after you made your move. I can’t think of a reason this shouldn’t work well, so long as you can discretize your decisions, although it still has issues with being noisy and overcomplicated.
Another specific story: Scott recently (probably non-seriously) proposed the idea of ConTracked:
This is, in principle, a pretty similar idea! We make a coin that relates to the underlying value (the probability that an infrastructure project will satisfy customers), and then let the contractor work on optimizing that. There’s a boring finance-y issue that makes this a potentially bad idea – it’s better to offload risk from contractors onto city governments – but this makes a real attempt to address the other main issue common to contracting out infrastructure projects: They’re a monopsony (the government is the only customer for most infrastructure projects) with unclear requirements (some knowable in advance – what neighborhoods do we want the new subway line to connect? – And some only knowable during construction – we ran into a watermain when digging this planned subway line, do we want to move the watermain or the line, and if so where?)
The traditional way to solve this issue is to have a competent, fast-turnaround department of planners in city hall who can make good decisions in a reasonable amount of time. This story of how Madrid built an entire subway system in eight years at a ridiculously low budget is amazing and everyone should read it. In principle replacing decisions by specific humans with betting markets should work – the contractor could, when facing a decision like where to put the subway line, make a betting market that pays out if the value of the ConTracked goes up upon that decision being made – but in practice this would be hugely complicated, require massive amounts of liquidity, and would be incredibly difficult to make happen fast. And as the Madrid story tells us, fast turnaround is hugely important (decision lags of months or years commonly kill projects).
(Also, this is all assuming we have a good way to define “completed an adequate subway system” to a decentralized court, which may be possible but definitely isn’t easy).
On reflection, I think the common theme here is that the betting market always has to happen on a decision level abstracted one level above the underlying market. Companies can’t make decisions based on the value of their stock, because the stock has to move faster than the companies do. In an alternate world where companies could make a decision, see how it affected their stock price, and then undo it the stock tanked, the stock wouldn’t tank, because investors would know the company could undo its bad decisions. The stock market requires that companies have a lot of inertia in forming and executing their strategies in order to function. Similarly, setting up a betting market for chess (in either of our forms) required setting the market up to run an order of magnitude faster than the actual game.[5]
And in the infrastructure example, you can’t speed it up by setting up a betting market, because a betting market is, in effect, rival teams of planners fighting (in an efficient way) over who has the best plan. It’ll have to run an order of magnitude slower than any individual team of planners (although in principle the decisions it produces are as good as the best decision by any individual team). Just hiring one of these teams to run your planning department directly is an order of magnitude faster and makes decisions of comparable quality (well, unless you’re New York).
[1] Yes, there’s also a whole bunch of other things things investors trade, like commodities and currencies, but the same principle applies.
[2] Or if Elon Musk tweets the name of an unrelated company with a similar name.
[3] Or if a random subreddit decides to pump up the stock for the 🚀 memes.
[4] Although you might have to ban people from betting in both the White and Black players’ betting markets, since that could create some weird incentives.
[5] Another method of handling chess – if we have a superhuman AI – which for chess, we do – we can just run a betting market on how well it will evaluate each possible move, without looking until after we’ve made it. Again, this requires a higher-level actor in order to work.