As part of the 2022 ACX prediction contest, which I am co-running with Eric Neyman, I wanted to get market prices for all of the predictions.[1] These market prices will be included with the contest data, which should be released soon.
This ended up being harder than I expected, and I can't guarantee this data is error-free. So in the interest of transparency, I'm making this post to explain where all the numbers came from.
Before I get into it, some notes:
These are supposed to be market prices as of February 14, 2022, which was the last day for contest submissions.
In some cases (like when backing a probability out of a Metaculus probability distribution) I didn't know how to get historical data. If it didn't look to me like the market had moved much since 2/14, I instead used the predictions as of today, March 5.
When aggregating predictions from different markets, I'm weighting them according to how many stars they get from Metaforecast, when available. When not available, I'm using my judgement.
Many of the Manifold markets were very lightly traded, some having only 1 or 2 bets. I tried to note this below when I noticed it.
There was some personal judgement involved in e.g. deciding whether a market asked close enough to the right thing to be relevant, adding or subtracting a few percentage points to PredictIt markets which don't sum to 100%, etc. If it seems like I misjudged anything here, it's possibly because I did.
I also expect that there's at least one outright error from something silly like copying the wrong number.
So: if you see anything wrong please let me know! Also please let me know if there's a relevant market which I missed or if you see a way to back predictions out of financial markets which I didn't think of (I've noted such cases below).
Anyway, here we go:
Scott predictions
Biden approval rating (as per 538) is greater than fifty percent: 14 (12 from Manifold; also this market from Kalshi saying 20% for having approval >45% on election day in November)
At least $250 million in damage from a single round of mass protests in US: 36 (Manifold)
PredictIt thinks Joe Biden is most likely 2024 Dem nominee: 73 (Manifold)
PredictIt thinks Donald Trump is most likely 2024 GOP nominee: 60 (Manifold)
Major flare-up (worse than past 5 years) in Russia/Ukraine conflict: 76 (88 from Manifold; 60% chance of invasion from Metaculus)
Major flare-up (worse than past 10 years) in Israel/Palestine conflict: 15 (Manifold)
Major flare-up (worse than in past 50 years) in China/Taiwan conflict: 15 (20 from Manifold; 10% chance of military conflict from Metaculus)
Honduran ZEDEs legally crippled to the point where no reasonable person would invest in them further: None (no one had yet made a bet in the Manifold market)
New ZEDE approved in Honduras: None (no one has made a bet in the Manifold marketeven now)
Gamestop stock price still above $100: 50 (GME options; Manifold said 24%, but there were 2 bets so I'm ignoring it)
Inflation for the year below five percent: 45 (Manifold; the financial markets probably have an opinion about this but I don't know how to find it - would appreciate any help!)
Unemployment below five percent in December: 73 (Kalshi said 92% that unemployment never goes above 6%; 49 from Manifold)
Google widely allows remote work, no questions asked: 14 (Manifold)
Bongbong Marcos will be elected president of the Philippines: 76 (80 from PredictIt; 75 from Metaculus; 63 from Manifold)
China will NOT reopen its borders at any point during the first half of 2022: 98 (Metaculus (market is technically about whether US travelers will need to quarantine); still no bets on Manifold)
Chinese GDP will continue to grow for the first 3/4 of the year: 95 (Manifold)
20% of US kids between 0.5 and 5 years old will get at least one COVID vaccine by year's end: 56 (50 from Metaculus; 68 from two bets on Manifold)
WHO will designate another Variant Of Concern: 76 (76 from Metaculus; 77 from Manifold)
12 billion COVID shots will be given out globally by November 1st: 92 (Metaculus; ignoring one bet from Manifold)
At least one country will have less than 10% of people vaccinated with two shots by November 1st: 81(Manifold; Metaculus has a market that only counts countries that are releasing data, so e.g. not North Korea; FP and Scott might resolve this one differently)
A psychedelic drug will be decriminalized/legalized in at least one more US state: 57 (59 from Metaculus; 55 from Manifold)
AI will discover a new drug promising enough for clinical trials: 71 (Manifold)
US government will NOT renew the ban on funding gain-of-function research: 73 (74 from Metaculus; 71 from Manifold)
The Biden administration will set the social cost of carbon at $100/ton or more: 35 (31 from Metaculus; 49 from two bets on Manifold)
2022 will be warmer than 2021: 70 (62 from Metaculus; 80 from Manifold)
Kenneth Branagh's Belfast will win Best Picture: 36 (35 from Metaculus; 37 from Manifold)
Matt Yglesias predictions
Democrats lose at least two Senate seats: 53 (63 from PredictIt; 48 from Metaculus; 45 eyeballed from Smarkets (their binning wasn't compatible with this question))
Democrats lose fewer than six Senate seats: 92 (PredictIt and Metaculus)
Nancy Pelosi announces retirement plans: 42 (Manifold; but maybe there's a way to back this out of other markets?)
Some version of USICA passes Congress: 80 (Kalshi)
Lula elected president of Brazil: 70 (67 from Polymarket; 72 from PredictIt)
China officially abandons Covid Zero: 74 (Manifold)
Additional booster shots of mRNA vaccines authorized for seniors: None (couldn't find any markets)
The Fed ends up doing more than its currently forecast three interest rate hikes: 84 (SOFR futures; see this comment by SimonM)
Viktor Orbán loses power in Hungary: 26 (17 from PredictIt; 20 from Smarkets; 45 from Metaculus on whether he loses the election; why the big difference here - is there concern that he would win the election and lose power anyway?)
Sinn Fein becomes the largest party in the Northern Ireland assembly: 80 (Smarkets)
Democrats lose at least one governor on net: 76 (Manifold)
17. Unemployment below five percent in December: 73 (Kalshi said 92% that unemployment never goes above 6%; 49 from Manifold)
I'm not sure exactly how you're converting 92% unemployment < 6% to < 5%, but I'm not entirely convinced by your methodology?
15. The Fed ends up doing more than its currently forecast three interest rate hikes: None (couldn't find any markets)
Looking at the SOFR Dec-22 3M futures 99.25/99.125 put spread on the 14-Feb, I put this probability at ~84%.
Thanks for doing this, I started doing it before I saw your competition and then decided against since it would have made cheating too easy. (Also why I didn't enter)
Re 17: You are right to be skeptical, because my methodology for this one was silly and ad hoc. I somewhat arbitrarily turned a 92% chance that unemployment never goes above 6% into a 80% chance that unemployment isn't above 5% in December. This is completely unprincipled, but I didn't have any better ideas, and the alternative was to ignore the Kalshi market completely and defer entirely to the 5 betters on Manifold, which seemed worse. If you have a more reasonable way of getting a number here, I'll happily defer to it.
As part of the 2022 ACX prediction contest, which I am co-running with Eric Neyman, I wanted to get market prices for all of the predictions.[1] These market prices will be included with the contest data, which should be released soon.
This ended up being harder than I expected, and I can't guarantee this data is error-free. So in the interest of transparency, I'm making this post to explain where all the numbers came from.
Before I get into it, some notes:
Anyway, here we go:
Scott predictions
(12 from Manifold; also this market from Kalshi saying 20% for having approval >45% on election day in November)
Future Perfect predictions
Matt Yglesias predictions
See also the excellent work of SimonM, who did this exercise for Scott's 2021 predictions.