In June 2012, the Association for Computing Machinery—a professional society of computer scientists, best known for hosting the prestigious ACM Turing Award, commonly referred to as the "Nobel Prize of Computer Science"—celebrated the 100th birthday of Alan Turing.
The event was attended by luminaries like, oh, in no particular order: Donald Knuth, Vint Cerf, Bob Kahn, Marvin Minsky, Judea Pearl, Ron Rivest, Adi Shamir, Leonard Adleman, and of extra relevance here; Ken Thompson, inventor of the UNIX operating system, co-inventor of the C programming language, and computer chess pioneer.
In all, some 33 Turing Award winners were scheduled to be in attendance. There were no parallel tracks or simultaneous panels going on.
So today, randomly watching one of the panel debates on Youtube, I was amazed by the amusing / horrifying example of failure of foresight and predictive accuracy, by these world leading computer scientists, regarding the advancement of the state of the art in artificial intelligence.
In this case the as it pertains to the ancient board game "go".
"When does a computer crack go?"
"And I will start by a 100 years, and then count down by ten year intervals."
And [by] 90 years? I count about 4% of the audience. (...)
Perhaps my internet searching skills are weak, but as best as I can tell, the incident has not been noted other than a few bemused Youtube comments in the video linked above.
Given ten options, ten buckets in which to place their bet, world-leading experts in computer science, as a group, managed to perform much worse than one would expect given their vast and wide-ranging expertise.
Worse even, than one would expect of a group of complete ignorants.
Given ten options, one would expect one out of every ten to land on the right answer, if nobody knew anything and everyone made a blind guess.
Three years and three months later, three-time European champion Fan Hui, was defeated. Half a year after that, world champion Lee Sedol was defeated by AlphaGo.
Interestingly, Ken Thompson, to whom the query was first directed, starts his answer by sharing an experience, from a World Computer Chess Championship, around 1980. Participants were asked if and when computers would beat the world champion at the game of chess. And, he explains, everyone except him alone, had been exceedingly optimistic.
Thereby priming the audience in several ways for the informal poll which followed.
By demonstrating authority, by proving a track record of sorts, by providing an anchor of sorts, and by warning against optimism. (Thompson had predicted that computers would beat human champions at chess by 2011.)
"Most of the predictions were like next year or five years.
You know, way, way optimistic.
If you look at Moore's law, on computers, and you look at the increase in speed. Ah, with strength, with speed, on computer chess,
never is just, you know, you can't predict never.
You just can't predict never.
It had to happen."
Moderator: "Do you think go then, is in the targets, of computers?"
Ah no, I don't think go is in... Ahh... If I had to predict go, I'd predict way, way out.
And then, the audience was polled, by simple show of hands.
One member of the audience stood alone, red-faced, to laughter and ridicule from the most esteemed peers in his field.
One single member out of the whole audience got it right.
- MP4 Video and Panel description at ACM.org: https://dl.acm.org/citation.cfm?id=2322182
- Video by ACM on Youtube at: https://youtu.be/dsMKJKTOte0?t=54m
- ACM Turing Centenary celebration program:
- https://web.archive.org/web/20120617021752/http://turing100.acm.org:80/finalprogram/tcc final_program.pdf
- ACM Turing 100 website: http://turing100.acm.org/ (dead link)
- AlphaGo on Wikipedia: https://en.wikipedia.org/wiki/AlphaGo