Since the human brain is not capable of recursive alteration of it's source code, and remains almost identical to the first conscious brains evolved 100,000 years ago, one must wonder if it is a tool capable of (or appropriate for) designing a friendly AI. In a time when the parabolic rate of increase in information far exceeds any possibility for natural selection to produce brains that do not rely on the evolved emotions and motivations you discuss, how can such a brain be expected to program the AI source code appropriately, when that brain is not capable of doing the same for itself? That is, how can that brain be expected to be capable of choosing what actually is "friendly", in light of its evolved state?
I agree with Anonymous: hire the talent. It's been done before, with proven results: the Manhattan project. In 1932, Einstein said: "There is not the slightest indication that nuclear energy will ever be obtainable". Ten years later it was solved, but only by a consortium of the brightest physicists, mathematicians, and engineers. No one person or specialty could do the job. They worked in parallel, like the brain. I suppose it has occurred to those working in AI that if it is what they say, and it is developed in the USA, it will become government property under eminent domain because of its effect on national security; so, the government might as well finance it's development. They have the money.
By the way, another Einstein quote, about an intelligent younger colleague: "Of course he's not humble; he hasn't accomplished anything yet."
âThere is not the slightest indication that nuclear energy will ever be obtainable. It would mean that the atom would have to be shattered at will.â â Albert Einstein, 1932
Ten years later, when all the greatest Western minds had been hired to work together in the Manhattan Project, the problem was solved.
Surely what I am about to write is obvious, and probably old. During World War II, when physicists began to realize the destructive potential of nuclear weapons, Albert Einstein was chosen by his peers to approach President Roosevelt. Einstein was perhaps not the best informed of the group, but he was the best known, and was thought to be able to get Roosevelt's ear, as he did. In response, Roosevelt was able to convene all the greatest Western minds in physics, mathematics, and engineering to work together for a rapid solution to the problem. Clearly, the importance of the development of recursively self-improving super-human intelligence has got to be, almost by definition, greater than all other current problems, since it is the one project that would allow for the speedy solution of all other problems. Is there no famous person or persons in the field, able to organize his peers, and with access to the government such that an effort similar to the Manhattan Project could be accomplished? The AI Institute has one research fellow, and are looking for one more. They have a couple of fund-raisers, but most of the world is unaware of AI altogether. This won't get it done in a reasonable time-frame. Your competitors may well be backed by their governments.
While the eventual use of the Manhattan Project's discoveries is about as far from Friendly AI as imaginable, the power of super-human recursive AI is such that no matter by whom or where it is developed it will become the eminent domain of a government, much like the most powerful Cray computers. You might as well have their money and all the manpower right from the start, and the ability to influence it's proper use.
Can/will this be done?
I think it would be cool if there were a mystery story whose sleuth-protagonist made explicit use of stats and probability.
Have you ever tried to get a book published? It's all about numbers, and I don't mean the kind you mention. What do you suppose is the potential audience for such a book? As Bob Unwin points out, the readers of this blog are not in any way indicative of the general public.