For the past year, I've been eagerly awaiting the start of 2018. This wasn't out of any dissatisfaction with 2017, or because of any specific events that I expected to happen today. No, I was excited for 2018 because it meant I got to grade my 2017 predictions and make predictions for 2018.
I've found this whole process great fun and would like to try and spread the habit. This post will include my graded 2017 predictions, templates so that you can make your own predictions, my calibration calculations, and my new predictions for 2018.
If you want to be able to look forward to the start of 2019 like I looked forward to today, or if you want to be able to make a cool graph (like this one!)
I recommend trying out my 2018 prediction templates before reading (to avoid being anchored by anything I write). The templates focus on international events and come in two versions:
With both these sheets, the idea is to pick a limited number of probabilities (I recommend 51%, 60%, 70%, 80%, and 90%) and assign one to each item that you have an opinion on. At the end of the year, you count the number of correct items in each probability bin and use that to see how close you were to ideal. This gives you an answer to the important question: "when I say something is 80% likely to happen, how likely, really, is it to happen?"
You can also make your own (or use the set of questions Slate Star Codex normally uses). If you do make your own, please link your post (and maybe also your template?) in the comments or post it to the front page. It's my hope that this post can serve as a convenient place for the LW community to look at the predictions of everyone who wants to participate in this experiment!
The following are my predictions for 2017, which were posted here.
15. Trump will veto at least 1 bill passed by the House and Senate – 70%
16. Changes to NAFTA will not significantly affect Canada (e.g. introduce tariffs, eliminate visas, etc) – 80%
17. Less than 100km of concrete wall on the border with Mexico will be constructed – 80%
18. Unemployment rate changes by less than 0.5% in 2017 – 90%
19. Bay Area housing prices increase in 2017 – 90%
20. Protests (in America) on Trump’s inauguration day draw at least 1 million people – 80%
21. Protests (in America) on Trump’s inauguration day draw at least 5 million people – 50%
22. Protests (in America) on Trump’s inauguration day draw less than 10 million people – 70%
23. Protests outside of America on Trump’s inauguration day draw at least 1 million people – 60%
24. Terrorist attack in America that kills at least 10 Americans – 70%
25. No terrorist attack in America that kills at least 100 Americans – 70%
26. No registry of Muslims created in America – 90%
27. New Supreme Court Justice is named to the USSC – 90%
28. No repeal of any of: the individual mandate, the prohibition on denying coverage for pre-existing conditions, children remaining on their parents insurance plans until they are 25 – 70%
29. Heathcare.gov is taken offline or otherwise rendered inoperative by the new administration – 80%
30. No Federal Department is eliminated – 80%
31. No setback to the FARC peace deal significant enough to cause >1000 rebels to rearm – 70%
32. On the black market, the exchange rate for Venezuelan Bolivars to US Dollars remains above 3000 bolivars per dollar. (As measured by DolarToday) – 80%
33. Inflation in Venezuela for 2017 is higher than 100% (As measured by DolarToday) – 90%
34. United Socialist party retains control of the Venezuelan presidency – 70%
35. No uprising in Venezuela leading to >1000 combined civilian and soldier deaths – 70%
36. The “Regulation” Bill, legalizing many illegal settlements, is passed in Israel – 60%
37. No Israeli politician is indicted by the ICC over settlement activity in 2017 – 80%
38. The US moves its embassy to Jerusalem – 50%
39. OPEC agreement fails (as evidenced by Saudi Arabia increasing oil production to >10.058 million BPD) – 50%
40. Iraq takes back Mosul – 90%
41. Mosul Dam does not fail – 70%
42. Fewer casualties in Syrian Civil War in 2017 than in 2016 – 60%
43. No new international sanctions against Iran – 80%
44. No new US sanctions against Iran – 50%
45. No attack on the Iranian nuclear program by Israel – 80%
46. Iran does not withdraw from the deal limiting its nuclear program – 80%
47. Conditional on Iran remaining in the nuclear deal, inspectors find no evidence of violations after the deal began – 90%
48. Yemen Civil War continues – 60%
49. Power transition in The Gambia requires ECOWAS troops – 50%
50. Power transition occurs in The Gambia – 70%
51. No peace deal ends South Sudan fighting – 50%
52. IS or affiliated groups do not hold more territory in Africa at the end of 2017 than at the beginning – 90%
53. Libya has a single government by the end of 2017 – 50%
54. No protests, riots, or rebellion in Egypt that kills >100 people in a one week period – 80%
55. No protests, riots, or rebellion in Tunisian kills >50 people in a one week period – 90%
56. At least one terrorist attack kills >50 people in Tunisian – 50%
57. Inflation rate in Japan remains below 1% in 2017 – 70%
58. No Japanese snap election in 2017 – 90%
59. Scandal involving Thailand’s new king makes its way to a major Western Newspaper – 50%
60. Saenuri Party loses in the 2017 South Korean election – 80%
61. China will send at least one diplomatic “insult” to the US (e.g. expelling an ambassador or consul or closing on of its embassies or consulates) – 60%
62. By the end of 2017, none of the young lawmakers associated with the Umbrella Revolution will be in the Hong Kong parliament – 60%
63. The Hong Kong lawmakers who are appealing their ban from parliament will have their final appeals denied – 80%
64. China will not deploy its military against either Hong Kong or Taiwan in 2017 – 90%
65. North Korea detonates a nuclear weapon – 70%
66. North Korea does not demonstrate a completed weapon system (e.g. miniaturized bomb and ICBM capable of threatening the continental United States) – 90%
67. No resolution to the crisis in Ukraine – 70%
68. Crimea remains part of Russia – 90%
69. Russian GDP growth is less than 2% – 80%
70. No gain of greater than 15% in the value of the ruble vs the dollar – 60%
71. Angela Merkel remains Chancellor of Germany – 60%
72. Marie Le Pen does not become President of France – 70%
73. Geert Wilders does not become Prime Minister of the Netherlands – 70%
74. UK invokes Article 50 – 60%
75. Conditional on the UK invoking article 50, this occurs behind schedule – 70%
76. Conditional on the UK leaving the EU, Scotland prepares for another referendum – 80%
77. No snap election called in the UK – 80%
78. No regional independence movement (e.g. Scotland, Catalan) achieves success in Europe in 2017 – 90%
79. Sanctions against Russia are not significantly rolled back (e.g. sanctions remain in place against Rosneft, Novate, Gazprombank and Vnesheconombank by all members of the G7 remain in place at the end of 2017) – 60%
80. I will not break up with anyone I am currently dating – 90%
81. I will buy a car – 50%
82. I will still be working at my current job at the end of 2017 – 80%
83. I will not move to another city in 2017 – 90%
84. Conditional on remaining in my current city, I will not move to a different apartment in 2017 – 80%
85. I will read at least 40 books this year – 80%
86. I will read at least 10 non-fiction books this year – 50%
87. I will start reading (and read at least 50 pages) of at least 10 books people recommended to me this year – 60%
88. I will write at least 200,000 words this year – 80%
89. I will post at least 15 blog posts or short stories – 80%
90. I will post at least 25 blog posts or short stories – 50%
91. I will be >15% over or under-confident for at least 2 confidence levels in these predictions (before taking into account this prediction) – 80%
I thought that making my predictions mostly numerical would make them easy to grade. This mostly worked, but there were a few edge cases, judgement calls, and other amusing things that I want to explicitly mention:
A complete list of the sources I used when grading all non-personal predictions is available here.
The whole point of having predictions with few allowed probabilities (for me it was 50%, 60%, 70%, 80% and 90%) is that you can then check how accurate these were by pooling your answers. Here's how I did:
Here's the graph from above again. The red line shows what I would get if I was a perfect judge of probability. The blue line is actual me. Whenever the red line is below my results, I was under-confident. Whenever it's above them, I was overconfident.
I'm pleased that in general (excepting 70% vs. 80%), things I thought were more likely were in fact more likely. I appear to be fairly under-confident at lower probability levels (50% through 70%), and fairly good at higher confidence levels (80% and 90%), although of course this is just one year and some of this could be due to chance and luck.
My meta-calibration was quite poor. I was never more than 10% off from perfect calibration, despite my worries that I would frequently be up to 15% from it.
I tried to keep in mind the results of my calibration while making these predictions. I've also made one slight change. Instead of using 50%, I'm using 51%, which has the implication that whichever way the question is phrased is the way I expect things to be more likely to turn out.
The formatting didn't come through when importing this post from your blog- especially the strikethroughs of failed predictions and the graphs!
What news sources would you recommend for increasing my awareness of global events?
Anything from outside whichever country you live in that has good global reach. The Guardian, Al Jazeera English, BBC, and Reuters are all good options.
I've found that the important thing is to check the news often. You can use one source for this, because the goal is just to find out what interesting things are going on. Then when there's a story you care about, you can get it deliberately from multiple angles.
Here are some good pairs of news sources:
Al Arabiya and Al Jazeera (Middle East)
Al Jazeera and Jerusalem Post (Middle East)
Jerusalem Post and Haaretz (Israel)
Briebart and Vox (America)
The Daily Mail and The Guardian (The UK)
Fars and literally anything western (Iran/Middle East)
Russia Today and literally anything western (Russia)
It's also really worth it to try and find a few local news sources (within the country of origin) for any story that really catches your interest. Especially reading a wide range of opinion pieces will give you a sense of what people think (although note the selection effect that will occur if you're reading English language news from a country that isn't English speaking; there will probably be some important difference between the people who speak English and the non-English speaking majoirty)
Fixed the image (kind of; there's a broken image icon stuck at the top that doesn't show up in the editor). The strikethrough shows up in the editor but not here. I've opened a bug for it.
Earlier, it seemed that the karma for this post reset when I edited it. But that didn't happen when I edited it this time. Could someone in the know chime in if karma is reset by edits, or if I was just confused by e.g. people removing upvotes?
I fixed the broken image thing, and am deploying a fix to the strikethrough not rendering right now. Should be up in the next 10 minutes or so. Sorry for the confusion!
The karma thing seems worrying. Karma should definitely not reset by edits. Do you know at what karma level your post was at earlier? There might be a bug here.
If I remember correctly it went from 7 to 3 when I edited the first time. But if that represents the difference between one and zero upvotes, then maybe it was just coincidence?
Yeah, that does sound more like coincidence. That would have been a single vote.