LESSWRONG
LW

by Stuart_Armstrong
1 min read8th Apr 20209 comments

36

World Modeling
Frontpage

Thanks everyone who volunteered! No longer accepting new volunteers now.

Results coming soon.

First of all, I want to thank all the volunteers who agreed to do this assessment. It's important to check the track record, especially in a period when expert predictions are so important and so debated. If you have any questions, add a comment here or PM me on LessWrong.

Logistics:

  • Go to this Google form (asking for your name/pseudonym and email) and I will send you a TSV (tab-separated values) file with all 105 questions in random order.
  • If you don't want to share your email with me, or share it on a Google form, send me a PM on LessWrong and we'll see what else we can do.

Assessment:

  • For each prediction, please answer on a five point scale: 1=True, 2=Weakly True, 3=Cannot decide, 4=Weakly False, 5=False. Use your intuitive assessment of Kurzweil's words if there is ambiguity in them.
  • Use your best estimate for the accuracy of the prediction only; do not consider how specific or trivial the prediction is. "People will breath oxygen" is true; "there will be a major pandemic in 2018, driving people to quarantine and crashing economies and stock markets" is false or weakly false, because of the specific date. If it said "around 2018", then it would be true or weakly true.
  • The prediction is true if it was true in 2019; thus, ignore any recent effects of COVID-19.

Procedure:

  • Do predictions on the .tsv file I sent you, adding another column.
  • Do them in the order of that file, starting from the top.
  • Try and attempt at least 10 predictions.
  • Some predictions start with a piece in square brackets []. This is to provide context for the prediction. Ignore it when assessing the accuracy.
  • I'll include the ordered list of predictions as a PDF. Use this if you need yet more context on the prediction.
  • Once you're done, send me the file back, and I will acknowledge receiving it. If you don't get an acknowledgement in the next two working days, ping me to see if I got it.

Cheers!

World Modeling
Frontpage

36

Mentioned in
Assessing Kurzweil's 1999 predictions for 2019
8th Apr 2020
New Comment
9 comments, sorted by
top scoring
Click to highlight new comments since: Today at 5:17 AM
[-]orthonormal4y100

Boy, lots of the remote learning stuff does suffer from predicting 2019 instead of 2020...

I suggest running inter-rater sanity checks and sometimes asking the rater if their answers should be inverted; I bet that at least one reader will misremember the instructions as "how true on a scale of 1 to 5" instead of your intended "how false on a scale of 1 to 5".

Reply
[-]moses4y100

lots of the remote learning stuff does suffer from predicting 2019 instead of 2020

I wouldn't call it a successful prediction anyway—he predicts this to be the normal state of affairs, whereas the current situation is a temporary reaction to extraordinary circumstances

Reply
[-]Stuart_Armstrong4y20

Boy, lots of the remote learning stuff does suffer from predicting 2019 instead of 2020...

I know. I hesitated, but decided to go for 2019 because we're unsure what will persist beyond 2020, of the current changes.

Reply
[-]Stuart_Armstrong4y20

Yep. I'll find questions with near consensus, and check whether some people are systematically inverting those.

Reply
[-]moses4y30

Wow, I haven't read the book so this was the first time encountering the predictions. They were… surprisingly bad, given it's just 20 years. Even adjusting for the fact that predictions tend to be ridiculous in general, this really exceeded all expectations

Reply
[-]Jsevillamol4y10

I do not understand prediction 86.

In other words, the difference between those "productively" engaged and those who are not is not always clear.

As context, prediction 84 says

While there is sufficient prosperity to provide basic necessities (secure housing and food,
among others) without significant strain to the economy, old controversies persist
regarding issues of responsibility and opportunity.

And prediction 85 says

The issue is complicated by the
growing component of most employment's being concerned with the employee's own
learning and skill acquisition.

What is Kurzweil talking about? Is this about whether we can tell when employees are doing useful work and when they are shirking?

Reply
[-]Stuart_Armstrong4y30

Is this about whether we can tell when employees are doing useful work and when they are shirking?

That's how I'd read it.

Reply
[-]Jsevillamol4y10

Sorry for being dense, but how should we fill it?

By default I am going to add a third column with the prediction, is that how you want to receive the data?

Reply
[-]Stuart_Armstrong4y30

Yes, that's it ^_^ Sorry if it wasn't clear; will reword.

Reply
Moderation Log