Agreed. AGI requires Judea Pearl more than it requires John Carmack.

AGI requires John von Neumann or Alan Turing or the like. Any of them would have decent programming expertise today.

The AGI requires something, that would also result in acquisition of familiarity with the tool-set of mankind, including the actual use of computers for reasoning, which requires you to be able to program. It is enough that the programming expertise might be useful, for the upcoming AGI insight maker to become a good programmer.

Who's likely to write the AI? A hypothesis

by NancyLebovitz 1 min read16th Feb 201128 comments


The idea of creating an AI seems to be getting more common.

To the extent that creating an AI is made easier by having more resources rather than by having more carefully thought out philosophy, the first AI will be created by a government or a business, not SIAI. I think the more resources side is the way to bet, but I'm open to argument.

If this is correct, the best strategy for Friendliness may be to keep working on the philosophy but not expect to code, and publicize the risks of Unfriendliness, both seriously and humorously.

The latter is based on something Scott Adams said (for what that's worth)-- that no one ever realizes they're the pointy-haired boss, but if anyone says "that plan sounds like something out of Dilbert", the plan is immediately taken out of consideration.

The good news, such as it is, is that the mistakes likely to be made by corporations and governments can be presented as funnier (or at least more entertaining to people who already dislike those institutions) than those likely to be made by people who are unthinkingly trying to create utopia.

ETA: It's conceivable that a large organization could have SIAI folks heading its AI project, but this doesn't seem likely.