This will be done unblinded, because Kurzweil's predictions are so well known that it would be infeasible to find large numbers of people who are technologically aware but ignorant of them.
Is this true? It could be, or alternatively it could simply appear true from your perspective of familiarity. I'm only vaguely aware of Kurzweil and have never heard any mention of him among my group of largely grad student / geek friends.
I don't think it's true. I think it would be pretty easy to survey people and include some questions checking for technological awareness and Kurzweil awareness, and quietly discarding any results from people low on the former or high on the latter.
I mean, you could do it on Amazon Mechanical Turk! Such people are pretty technically sophisticated to be on Mechanical Turk in the first place, psychologists use it for surveys all the time, and a dismayingly large fraction of Turkers have college educations. It'd work fine.
http://www.kurzweilai.net/predictions/download.php
This contains all his predictions, it shouldn't be hard to verify them.
I'm likewise vaguely aware of Kurzweil's reputation and accomplishments, and vaguely recall mention of a big prediction in the field of AI, a very optimistic one according to critics IIRC.
That's about the extent of it. I wasn't aware he had publicly made other predictions.
Problem is, how would one go about verifying this? Not to mention that if I wasn't already primed to not click the link, I would probably have immediately searched for the predictions in question just to know the subject under discussion.
On that note, thanks and good move Stuart, warning us beforehand about the spoiler/unblinding link.
I doubt it's true. I think it would be relatively easy to find technically sophisticated people who're unaware of Kurzweil's specific predictions; it'd be harder to find technically sophisticated people who're consistently unaware of his general thesis, but I'll bet you could still do it. You'd just need to look outside the transhumanist/singularitarian/AI enthusiast cluster.
Since those clusters are pretty tightly grouped in terms of conceptual underpinnings, it should be easy to filter them from a sample. Getting a good sample would be harder -- LW wouldn't do it, and personal blogs wouldn't either. Gwern's idea below looks promising but I have no idea how you'd go about it.
I will do 10 predictions. Email mtgandp gmail.
Interestingly, Stuart seems to have adopted a similar approach to the synagogue described in Why Our Kind Can't Cooperate, with people publicly declaring their commitments.
One way to gauge how reliable people's judgements are: have multiple people rate each Kurzweil prediction and see how well their ratings agree. So far LWers have committed to checking at least 200 predictions, so if everyone pulls through you'll be able to get multiple ratings of at least 28 questions. Those multiple ratings could then be cross-checked for each question.
(I won't volunteer to rate any statements myself because (1) I'm lazy; (2) I already have a mildly negative view of Kurzweil's predictive ability, which might make me biased; and (3) I read your earlier post and re-rated the 10 Age of Spiritual Machines predictions in that post myself, so I've already been primed in that respect.)
One way to gauge how reliable people's judgements are: have multiple people rate each Kurzweil prediction and see how well their ratings agree.
This is a good idea. It's standard operating procedure (for measures which require a rater's judgment) to have 2 raters for at least some of the items, and to report the agreement rate on those items ("inter-rater reliability"). Be sure to vary which raters are overlapping; for example, don't give gwern and bsterrett the same 10 predictions (instead have maybe one prediction that they both rate, and one where bsterrett & Tenoke overlap, etc.) - that way the agreement rate tells you something about how much agreement there is between all of the raters (and not just between particular pairs of raters).
In cases where the 2 raters disagree, you could just have a 3rd rater rate it and then go with their rating, or you could do something more complicated (like having the two raters discuss it and try to reach a consensus).
I'm up for that -- andrew @ thenationalpep.co.uk . I can probably do ten. Disclosure of biases upfront -- I'm not familiar with that specific book of Kurzweil, or with much of Kurzweil's work in general, so I don't know what his predictions are. But I am familiar with his book The Singularity Is Near, which I thought utterly, comprehensively wrongheaded to a point where most of what I could say about it would seem like personal abuse, so I suspect that a relatively low proportion of his predictions will come true.
I'm aware of Kurzweil but I don't think I've read anything of his except for your earlier fact checking article (and that was long enough ago that I don't remember a thing about it).
It wouldn't be so hard to add a "do you think this prediction was made by Kurzweil or someone else? (don't google, but if you accidentally found the answer while researching, don't pretend not to know, either)" checkbox.
Predictions are cheap and easy; verification is hard, essential, and rare. For things like AI, we seem to be restricted to nothing but expert predictions - but expert predictions on AI are not very good, either in theory or in practice. If we are some experts who stand out, we would really want to identify them - and there is nothing better than a track record for identifying true experts.
So we're asking for help to verify the predictions of one of the most prominent futurists of this century: Ray Kurzweil, from his book "The Age of Spiritual Machines". By examining his predictions for times that have already come and gone, we'll be able to more appropriately weight his predictions for times still to come. By taking part, by lending your time to this, you will be directly helping us understand and predict the future, and will get showered in gratitude and kudos and maybe even karma.
I've already made an attempt at this (if you are interested in taking part in this project, avoid clicking on that link for now!). But you cannot trust a single person's opinions, and that was from a small (albeit random) sample of the predictions. For this project, I've transcribed his predictions into 172 separate (short) statements, and any volunteers would be presented with a random selection among these. The volunteers would then do some Google research (or other) to establish whether the prediction had come to pass, and then indicate their verdict. More details on what exactly will be measured, and how to interpret ambiguous statements, will be given to the volunteers once the project starts.
If you are interested, please let me know at stuart.armstrong@philosophy.ox.ac.uk (or in the comment thread here), indicating how many of the 172 questions you would like to attempt. The exercise will probably happen in late November or early December.
This will be done unblinded, because Kurzweil's predictions are so well known that it would be infeasible to find large numbers of people who are technologically aware but ignorant of them. Please avoid sharing your verdicts with others; it is entirely your own individual assessment that we are interested in having.