For the past year, I've been eagerly awaiting the start of 2018. This wasn't out of any dissatisfaction with 2017, or because of any specific events that I expected to happen today. No, I was excited for 2018 because it meant I got to grade my 2017 predictions and make predictions for 2018.

I've found this whole process great fun and would like to try and spread the habit. This post will include my graded 2017 predictions, templates so that you can make your own predictions, my calibration calculations, and my new predictions for 2018.

If you want to be able to look forward to the start of 2019 like I looked forward to today, or if you want to be able to make a cool graph (like this one!)

I recommend trying out my 2018 prediction templates before reading (to avoid being anchored by anything I write). The templates focus on international events and come in two versions:

• Long (which assumes you read global news a lot)
• Short (which is less demanding)

With both these sheets, the idea is to pick a limited number of probabilities (I recommend 51%, 60%, 70%, 80%, and 90%) and assign one to each item that you have an opinion on. At the end of the year, you count the number of correct items in each probability bin and use that to see how close you were to ideal. This gives you an answer to the important question: "when I say something is 80% likely to happen, how likely, really, is it to happen?"

You can also make your own (or use the set of questions Slate Star Codex normally uses). If you do make your own, please link your post (and maybe also your template?) in the comments or post it to the front page. It's my hope that this post can serve as a convenient place for the LW community to look at the predictions of everyone who wants to participate in this experiment!

2017 Predictions

The following are my predictions for 2017, which were posted here.

Canada

1. Trudeau ends the year with a lower approval rating than he started – 60%
2. No bill introduced that changes the electoral system away from first past the post in 2019 – 50%
3. No referendum scheduled on changing the electoral system away from first past the post before 2019 – 70%
4. A bill legalizing marijuana is passed by the House of Commons – 90%
5. The senate doesn’t block attempts to legalize marijuana – 80%
6. At least one court finds the assisted dying bill isn’t in line with Carter v Canada – 60%
7. Ontario Liberal Approval rating remains below 30% – 80%
8. Patrick Brown “unsure” rating remains above 40% – 70%
9. Kellie Leitch is not the next CPC leader – 80%
10. Michael Chong is not the next CPC leader – 70%
11. Maxine Bernier is not the next CPC leader – 90%
12. No terrorist attack that kills >10 Canadians – 70%
13. No terrorist attack that kills >100 Canadians – 90%
14. At least one large technology company (valuation >$10 billion and >1,000 employees) will open a Waterloo office in 2017 – 80%

America

15. Trump will veto at least 1 bill passed by the House and Senate – 70%

16. Changes to NAFTA will not significantly affect Canada (e.g. introduce tariffs, eliminate visas, etc) – 80%

17. Less than 100km of concrete wall on the border with Mexico will be constructed – 80%

18. Unemployment rate changes by less than 0.5% in 2017 – 90%

19. Bay Area housing prices increase in 2017 – 90%

20. Protests (in America) on Trump’s inauguration day draw at least 1 million people – 80%

21. Protests (in America) on Trump’s inauguration day draw at least 5 million people – 50%

22. Protests (in America) on Trump’s inauguration day draw less than 10 million people – 70%

23. Protests outside of America on Trump’s inauguration day draw at least 1 million people – 60%

24. Terrorist attack in America that kills at least 10 Americans – 70%

25. No terrorist attack in America that kills at least 100 Americans – 70%

26. No registry of Muslims created in America – 90%

27. New Supreme Court Justice is named to the USSC – 90%

28. No repeal of any of: the individual mandate, the prohibition on denying coverage for pre-existing conditions, children remaining on their parents insurance plans until they are 25 – 70%

29. Heathcare.gov is taken offline or otherwise rendered inoperative by the new administration – 80%

30. No Federal Department is eliminated – 80%

South America

31. No setback to the FARC peace deal significant enough to cause >1000 rebels to rearm – 70%

32. On the black market, the exchange rate for Venezuelan Bolivars to US Dollars remains above 3000 bolivars per dollar. (As measured by DolarToday) – 80%

33. Inflation in Venezuela for 2017 is higher than 100% (As measured by DolarToday) – 90%

34. United Socialist party retains control of the Venezuelan presidency – 70%

35. No uprising in Venezuela leading to >1000 combined civilian and soldier deaths – 70%

Middle East

36. The “Regulation” Bill, legalizing many illegal settlements, is passed in Israel – 60%

37. No Israeli politician is indicted by the ICC over settlement activity in 2017 – 80%

38. The US moves its embassy to Jerusalem – 50%

39. OPEC agreement fails (as evidenced by Saudi Arabia increasing oil production to >10.058 million BPD) – 50%

40. Iraq takes back Mosul – 90%

41. Mosul Dam does not fail – 70%

42. Fewer casualties in Syrian Civil War in 2017 than in 2016 – 60%

43. No new international sanctions against Iran – 80%

44. No new US sanctions against Iran – 50%

45. No attack on the Iranian nuclear program by Israel – 80%

46. Iran does not withdraw from the deal limiting its nuclear program – 80%

47. Conditional on Iran remaining in the nuclear deal, inspectors find no evidence of violations after the deal began – 90%

48. Yemen Civil War continues – 60%

Africa

49. Power transition in The Gambia requires ECOWAS troops – 50%

50. Power transition occurs in The Gambia – 70%

51. No peace deal ends South Sudan fighting – 50%

52. IS or affiliated groups do not hold more territory in Africa at the end of 2017 than at the beginning – 90%

53. Libya has a single government by the end of 2017 – 50%

54. No protests, riots, or rebellion in Egypt that kills >100 people in a one week period – 80%

55. No protests, riots, or rebellion in Tunisian kills >50 people in a one week period – 90%

56. At least one terrorist attack kills >50 people in Tunisian – 50%

Asia

57. Inflation rate in Japan remains below 1% in 2017 – 70%

58. No Japanese snap election in 2017 – 90%

59. Scandal involving Thailand’s new king makes its way to a major Western Newspaper – 50%

60. Saenuri Party loses in the 2017 South Korean election – 80%

61. China will send at least one diplomatic “insult” to the US (e.g. expelling an ambassador or consul or closing on of its embassies or consulates) – 60%

62. By the end of 2017, none of the young lawmakers associated with the Umbrella Revolution will be in the Hong Kong parliament – 60%

63. The Hong Kong lawmakers who are appealing their ban from parliament will have their final appeals denied – 80%

64. China will not deploy its military against either Hong Kong or Taiwan in 2017 – 90%

65. North Korea detonates a nuclear weapon – 70%

66. North Korea does not demonstrate a completed weapon system (e.g. miniaturized bomb and ICBM capable of threatening the continental United States) – 90%

Europe

67. No resolution to the crisis in Ukraine – 70%

68. Crimea remains part of Russia – 90%

69. Russian GDP growth is less than 2% – 80%

70. No gain of greater than 15% in the value of the ruble vs the dollar – 60%

71. Angela Merkel remains Chancellor of Germany – 60%

72. Marie Le Pen does not become President of France – 70%

73. Geert Wilders does not become Prime Minister of the Netherlands – 70%

74. UK invokes Article 50 – 60%

75. Conditional on the UK invoking article 50, this occurs behind schedule – 70%

76. Conditional on the UK leaving the EU, Scotland prepares for another referendum – 80%

77. No snap election called in the UK – 80%

78. No regional independence movement (e.g. Scotland, Catalan) achieves success in Europe in 2017 – 90%

79. Sanctions against Russia are not significantly rolled back (e.g. sanctions remain in place against Rosneft, Novate, Gazprombank and Vnesheconombank by all members of the G7 remain in place at the end of 2017) – 60%

Personal

80. I will not break up with anyone I am currently dating – 90%

81. I will buy a car – 50%

82. I will still be working at my current job at the end of 2017 – 80%

83. I will not move to another city in 2017 – 90%

84. Conditional on remaining in my current city, I will not move to a different apartment in 2017 – 80%

85. I will read at least 40 books this year – 80%

86. I will read at least 10 non-fiction books this year – 50%

87. I will start reading (and read at least 50 pages) of at least 10 books people recommended to me this year – 60%

88. I will write at least 200,000 words this year – 80%

89. I will post at least 15 blog posts or short stories – 80%

90. I will post at least 25 blog posts or short stories – 50%

91. I will be >15% over or under-confident for at least 2 confidence levels in these predictions (before taking into account this prediction) – 80%

Notes

I thought that making my predictions mostly numerical would make them easy to grade. This mostly worked, but there were a few edge cases, judgement calls, and other amusing things that I want to explicitly mention:

• Throughout my predictions I used the word "remains". I regret this, because it is ambiguous. I think I intended it to mean "on January 1st, 2018, X remains true", but there's an alternative parsing that is "during all of 2017, X will be true". I feel it's most accurate to grade these according to my intent. For my 2018 predictions, I will use clearler language.
• For 8, the two most recent polls I could find were both from November. In one, Patrick Brown had a "don't know" rating of 50%. In the other, it was 34%. Polls were found by Googling 'ontario leader popularity' and 'ontario leader popularity politics'; November was the last month in which I could find polls, so I only used November polls. I'm averaging these two and considering the prediction successful. The lack of good aggregation of Ontario political information is part of why I would like to create a website tracking the Ontario election this year.
• While I was correct as to the who wasn't the next leader of the Conservative party, I definitely got emotionally involved such that I was severely miscalibrated. Bernier came far closer to winning it then Leitch or Chong and both of those two fringe candidates had much lower chances than it felt like they did.
• WRT 24 and 25, I'm not counting the Las Vegas shooting as a terrorist attack because it lacked a political motive (as far as we currently know). I think I overestimated the risk of terrorist attack because of the availability heuristic (the last two years had seen a higher than normal amount of successful and dangerous terrorist attacks). A proper estimation would have focused more on the base rate.
• I don't think Trump's cancellation of advertising comes anywhere close to fulfilling 29, so I'm marking it as failed.
• 31 is borderline, with many former FARC fighters "joining criminal gangs or a dissident FARC movement that has about 1,000 fighters nationwide". Given that this still implies less than 1000 members rearmed to continue the fight as FARC, I think the prediction holds.
• 38 is also borderline, but ultimately, I think there is a difference between an announcement of intent to move and an actual movement. Since I was going to mark 28 as a success if Trump hadn't signed the tax bill by now, it's only fair that I mark 38 as a failure.
• I was way under-confident in the stability of the Mosul dam (41). Compare my probability with the chance on the Good Judgement Project and you'll see I really overstated the risk compared to the consensus.
• WRT to 43 and 44, the US Treasury added new groups to existing designations, but these are neither new sanctions, nor international sanctions.
• For 59, I think this Daily Mail headline counts: "Thailand's colourful new King brought 'his mistress AND his former air stewardess wife' to his father's lavish cremation ceremony with both marching in bearskin hats". I think I want to stay away from predicting "scandals" in the future though, because it's a very fuzzy concept.
• While North Korea claims to have a complete, miniaturized ICBM, it looks like them actually realizing this with a weapon able to hit the US mainland is about one year away. Therefore 66 is a success.
• 69 is only provisionally true and needs to be revised when more GDP data is available.
• I am apparently rubbish at predicting snap elections, given that I got both 77 and 58 wrong, while being highly confident in my wrongness.
• Out of all of my failed predictions, the one that surprised me the most was the OPEC deal holding. I really thought that it would fall apart.

A complete list of the sources I used when grading all non-personal predictions is available here.

Calibration

The whole point of having predictions with few allowed probabilities (for me it was 50%, 60%, 70%, 80% and 90%) is that you can then check how accurate these were by pooling your answers. Here's how I did:

• Of my predictions at a 50% confidence level, I got 7 right and 6 wrong (54%).
• Of my predictions at a 60% confidence level, I got 9 right and 4 wrong (69%).
• Of my predictions at a 70% confidence level, I got 16 right and 4 wrong (80%).
• Of my predictions at an 80% confidence level, I got 20 right and 6 wrong (77%).
• Of my predictions at a 90% confidence level, I got 17 right and 2 wrong (89%).

Here's the graph from above again. The red line shows what I would get if I was a perfect judge of probability. The blue line is actual me. Whenever the red line is below my results, I was under-confident. Whenever it's above them, I was overconfident.

I'm pleased that in general (excepting 70% vs. 80%), things I thought were more likely were in fact more likely. I appear to be fairly under-confident at lower probability levels (50% through 70%), and fairly good at higher confidence levels (80% and 90%), although of course this is just one year and some of this could be due to chance and luck.

My meta-calibration was quite poor. I was never more than 10% off from perfect calibration, despite my worries that I would frequently be up to 15% from it.

2018

I tried to keep in mind the results of my calibration while making these predictions. I've also made one slight change. Instead of using 50%, I'm using 51%, which has the implication that whichever way the question is phrased is the way I expect things to be more likely to turn out.

Canada

1. Liberals remain ahead in the CBC Poll Tracker seat projection – 70%
2. Trudeau has a higher net favorability rating than Andrew Scheer according to the CBC Leader Meter on January 1, 2019 – 80%
3. Marijuana is legalized in time for Canada Day – 60%
4. Marijuana is legalized in 2018 – 90%
5. At least one court finds the assisted dying bill isn't in line with Carter v Canada – 70%
6. Ontario PC party wins the election – 60%
7. The Ontario election results in a minority government – 80%
8. The Quebec election results in a minority government – 80%
9. No BC snap election in 2018 – 90%
10. No terrorist attack in Canada that kills > 10 Canadians in 2018 – 90%
11. More Canadian opioid poisoning deaths in 2018 than in 2017 – 60%
12. Canada does better at the 2018 Winter Olympics (in both gold medals and total medals) than in 2014 – 90%
13. Canada does not win a gold medal in men's hockey at the 2018 Olympics – 70%
14. Canada does win a gold medal in women's hockey at the 2018 Olympics – 51%

America

1. Trump announces that the US is pulling out of NAFTA and begins the process of putting the US withdrawal into motion – 51%
2. Less than 100km of concrete wall on the border with Mexico will be constructed – 90%
3. No registry of Muslims created – 90%
4. Congress doesn't take action to extend DACA – 80%
5. No department of the Federal Government is eliminated – 90%
6. There isn't a government shutdown before the midterm elections – 60%
7. Democrats take back the house in the 2018 midterm elections – 80%
8. Democrats take back the senate in the 2018 midterm elections – 60%
9. Mueller's investigation finishes in 2018 – 60%
10. Impeachment proceedings aimed at Trump are not started in 2018 – 80%
11. Trump is still president at the end of 2018 – 90%
12. No terrorist attack in America that kills > 10 Americans – 70%
13. No terrorist attack in America that kills > 100 Americans – 90%
14. Susan Collins doesn't get the Obamacare stabilization measures she was promised – 70%
15. More US opioid poisoning deaths in 2018 than in 2017 – 80%

South America

1. FARC peace deal remains in place on January 1, 2019 – 80%
2. The black market exchange rate for Venezuelan Bolivars is above 110,000 to the US dollar on January 1, 2019 (as measured by DolarToday) – 80%
3. Inflation in Venezuela is above 100% for the year of 2018 (as measured by DolarToday) – 90%
4. United Socialist party retains control of the Venezuelan presidency in 2018 – 90%
5. Protests (and the official response to those protests) result in more than 100 fatalities in Venezuela in 2018 – 60%
6. Protests (and the official response to those protests) do not result in more than 1000 fatalities in Venezuela in 2018 – 70%
7. Major Venezuelan opposition groups do not enter any sort of power sharing agreement with the Venezuelan regime in 2018 – 80%

Middle East

1. No Israeli politician is indicted by the ICC over settlement activity in 2018 – 90%
2. There isn't an election in Israel in 2018 – 80%
3. US does not physically relocate its embassy to Jerusalem in 2018 – 90%
4. No Palestinian led Intifada in Israel that results in the deaths of >1000 combined attackers, security forces, and civilians (this is a conflict characterized by suicide bombing and police responses) – 70%
5. No Israeli led operation in the West Bank or Gaza that results in the deaths of >1000 combined soldiers, civilians, and militants (this is a conflict characterized by rocket fire and military strikes) – 70%
6. Fatah and Hamas do not meaningfully reconcile in 2018 (e.g. Fatah still doesn't control Gaza by January 1, 2019) – 51%
7. No significant resurgence in ISIL in 2018 (e.g. it does not gain territory over the next year) – 80%
8. Fewer casualties in the Syrian Civil War in 2018 than in 2017 – 70%
9. No power sharing agreement or durable ceasefire (typified by the three months following the agreement each having less than 500 fatalities) in Syria in 2018 – 80%
10. Bashar Al Assad is still President of Syria on January 1, 2019 – 90%
11. Protests in Iran do not result in more than 1000 fatalities by the end of 2018 – 70%
12. Protests in Iran do not result in more than 100 fatalities by the end of 2018 – 51%
13. Hassan Rouhani is still President of Iran on January 1, 2019 – 90%
14. No new international sanctions against Iran (does not include adding new organizations or individuals to old categories and requires coordinated participation of at least two countries) – 80%
15. No new US sanctions against Iran (does not include adding new organizations or individuals to old categories) – 51%
16. No attack on the Iranian nuclear program by Israel – 90%
17. Iran does not withdraw from the deal limiting its nuclear program – 90%
18. Conditional on Iran remaining in the nuclear deal, inspectors find no evidence of violations after the deal began – 90%
19. Yemen Civil War continues – 60%
20. Saudi Arabia pulls troops out of Yemen – 51%
21. Mohammed bin Salman either remains as crown prince of Saudi Arabia, or becomes king (i.e. no coup or succession shake-up) – 80%
22. Rockets fired from Yemen cause casualties in another country – 51%
23. No resolution or lifting of embargo in the Qatar crisis – 80%
24. OPEC production cuts continue through to the end of 2018 – 60%

Africa

1. No power sharing between ZANU-PF and the opposition will happen in Zimbabwe before the elections (if they occur) in 2018 – 80%
2. Zimbabwe will hold election in 2018 – 70%
3. No peace deal ends South Sudan fighting – 70%
4. Libya still has two rival governments on January 1, 2019 – 70%
5. No protests, riots, or rebellion in Egypt that kills >100 people in a one week period – 80%
6. No protests, riots, or rebellion in Tunisian kills >50 people in a one week period – 90%
7. No terrorist attack in Tunisia kills >20 people – 80%
8. Zuma is not impeached in 2018 – 51%

Asia

1. Inflation rate in Japan still remains below 1% in 2018 – 70%
2. Japanese constitutional reform (removing pacifism) does not occur in 2018 – 51%
3. China will not deploy its military against Taiwan or Hong Kong in 2018 – 90%
4. North Korea will test a submarine launched ballistic missile in 2018 – 70%
5. North Korea will not test nuclear weapons or launch any missiles during the 2018 Olympics – 80%
6. North Korea will test a nuclear weapon in 2018 – 51%
7. No country will attempt to shoot down a North Korean missile test in 2018 – 80%
8. If there is an attempt, it will succeed – 51%
9. North Korea tests a missile that is judged by experts at 38 North as likely able to carry a plausible North Korean nuclear weapon to the United States – 60%
10. No current member of China's Politburo Standing Committee visits North Korea in 2018 – 70%
11. No meeting between Kim Jung-un and Moon Jae-in in 2018 – 90%

Europe

1. No resolution to the crisis in Ukraine – 80%
2. Russian GDP growth is less than 3% – 80%
3. No gain of greater than 20% in the value of the ruble vs. the dollar – 70%
4. Sanctions against Russia are not significantly rolled back (e.g. sanctions remain in place against Rosneft, Novate, Gazprombank and Vnesheconombank by all members of the G7 remain in place at the end of 2018) – 90%
5. Angela Merkel remains chancellor of Germany – 60%
6. Germany holds another election before a government can be formed – 51%
7. No date set for another Scottish referendum in 2018 – 80%
8. Teresa May remains prime minister of the United Kingdom – 70%
9. The UK does not terminate the process of Brexit in 2018 – 90%
10. There is no final Brexit withdrawal deal reached in 2019 (Donald Tusk wishes to have one by October) – 51%
11. No snap election/vote of no-confidence in the UK in 2018 – 80%
12. Poland's EU voting rights aren't suspended – 90%
13. Poland and Hungary continue to refuse to accept migrant quotas – 90%

Notes

• I originally made predictions after I was inspired by the yearly predictions on Slate Star Codex. Thanks Scott!
• I had some problems with the list numbering restarting here. If you want to see everything properly numbered, you can check out the original here
• I'm not going to make any edits to this post (even to fix typos) after January 8th.
• I'd like to thank Tessa Alexanian for giving advice on the layout and title.