Similar to the monthly Rationality Quotes threads, this is a thread for memorable quotes about Artificial General Intelligence.
The best answer to the question, "Will computers ever be as smart as humans?” is probably “Yes, but only briefly.”
It is seemingly easy to get stuck in arguments over whether or not machines can "actually" think.
It is sufficient to assess the effects or outcomes of the phenomenon in question.
By sidestepping the question of what, exactly, it means to "think",
we can avoid arguing over definitions, yet lose nothing of our ability to model the world.
Does a submarine swim? The purpose of swimming is to propel oneself through the water. A nuclear powered submarine can propel itself through the oceans at full speed for months at a time. It achieves the purpose of swimming, and does so rather better than a fish, or a human.
If the purpose of thinking is isomorphic to:
Model the world in order to formulate plans for executing actions which implement goals.
Then, if a machine can achieve the above we can say it achieves the purpose of thinking,
akin to how a submarine successfully achieves the purpose of swimming.
Discussion of whether the machine really thinks is now superfluous.
"Sorry Arthur, but I'd guess that there is an implicit rule about announcement of an AI-driven singularity: the announcement must come from the AI, not the programmer. I personally would expect the announcement in some unmistakable form such as a message in letters of fire written on the face of the moon." - Dan Clemmensen, SL4
We probably make thousands of species extinct per year through our pursuit of instrumental goals, why is it so hard to imagine that AGI could do the same to us?
Machines can within certain limits beget machines of any class, no matter how different to themselves... Complex now, but how much simpler and more intelligibly organised may [a machine] not become in another hundred thousand years? or in twenty thousand? For man at present believes that his interest lies in that direction; he spends an incalculable amount of labour and time and thought in making machines breed always better and better; he has already succeeded in effecting much that at one time appeared impossible, and there seem no limits to the results of accumulated improvements if they are allowed to descend with modification from generation to generation. It must always be remembered that man’s body is what it is through having been moulded into its present shape by the chances and changes of many millions of years, but that his organisation never advanced with anything like the rapidity with which that of the machines is advancing. This is the most alarming feature in the case, and I must be pardoned for insisting on it so frequently.
Samuel Butler (1872)
The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.
Eliezer Yudkowsky (2008)
Though we have to live and work with (and against) today's mechanical morons, their deficiencies should not blind us to the future. In particular, it should be realized that as soon as the borders of electronic intelligence are passed, there will be a kind of chain reaction, because the machines will rapidly improve themselves... there will be a mental explosion; the merely intelligent machine will swiftly give way to the ultraintelligent machine.... Perhaps our role on this planet is not to worship God but to create Him.
Arthur C. Clarke (1968)
"There are lots of people who think that if they can just get enough of something, a mind will magically emerge. Facts, simulated neurons, GA trials, proposition evaluations/second, raw CPU power, whatever. It's an impressively idiotic combination of mental laziness and wishful thinking." - Michael Wilson
If the Singularity can not be prevented or confined, just how bad could the Post-Human era be? Well ... pretty bad.
... in original.
I once remarked that to design ultraintelligent machines was to play with fire, that we had played with fire once before, and it had kept the other animals at bay. Arthur Clarke's reply was that this time we are the other animals.
The machines will do what we ask them to do and not what we ought to ask them to do.
"In the universe where everything works the way it common-sensically ought to, everything about the study of Artificial General Intelligence is driven by the one overwhelming fact of the indescribably huge effects: initial conditions and unfolding patterns whose consequences will resound for as long as causal chains continue out of Earth, until all the stars and galaxies in the night sky have burned down to cold iron, and maybe long afterward, or forever into infinity if the true laws of physics should happen to permit that. To deliberately thrust you...
We must develop as quickly as possible technologies that make possible a direct connection between brain and computer, so that artificial brains contribute to human intelligence rather than opposing it.
Compare any kind of machine you may happen to think of with what its ancestor was only twenty-five years ago. Its efficiency has doubled, trebled... By knowledge alone man might extinguish himself utterly... Man's further task is... to learn how best to live with these powerful creatures of his mind [the machines], how to give their fecundity a law and... how not to employ them in error against himself.
Garet Garrett (1926)
The panel of experts was overall skeptical of the radical views expressed by futurists and science-fiction authors. [...] There was overall skepticism about the prospect of an intelligence explosion as well as of a “coming singularity,” and also about the large-scale loss of control of intelligent systems. [...] The group suggested outreach and communication to people and organizations about the low likelihood of the radical outcomes, sharing the rationale for the overall comfort of scientists in this realm
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion”, and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make.
Good (1965)
...we are ourselves creating our own successors; we are daily adding to the beauty and delicacy of their physical organisation; we are daily giving them greater power and supplying by all sorts of ingenious contrivances that self-regulating, self-acting power which will be to them what intellect has been to the human race. In the course of ages we shall find ourselves the inferior race... the time will come when the machines will hold the real supremacy over the world and its inhabitants.
Samuel Butler (1863)
The mathematician John von Neumann, born Neumann Janos in Budapest in 1903, was incomparably intelligent, so bright that, the Nobel Prize-winning physicist Eugene Wigner would say, "only he was fully awake." One night in early 1945, von Neumann woke up and told his wife, Klari, that "what we are creating now is a monster whose influence is going to change history, provided there is any history left. Yet it would be impossible not to see it through." Von Neumann was creating one of the first computers, in order to build nuclear weapons. But, Klari said, it was the computers that scared him the most.
Konstantin Kakaes
"It seems inevitable that sometime in this century, Moore's Law combined with greater understanding of intelligence itself will drive machine intelligence to levels beyond, and soon thereafter, beyond anything we can imagine. When intelligent machines begin designing superintelligence machines and their software, intelligence should grow exponentially. The result could be a runaway intelligence explosion.
T.M. Georges (2004)
...Most people are strongly biased toward not wanting a computer to be able to think. Why? For a variety of reasons, the layperson's concept think has become so intertwined with the concept human that many people have an emotional reaction against the idea of nonhuman things thinking...
However, despite their strong feelings against the idea of thinking computers, most people have not thought about the issue very carefully and are at a loss to come up with a definition of thinking that would include most humans (babies, for example) and exclude all computers.
Why we're doomed reason #692...
Here is Hugo de Garis, in the opening of The Artilect War:
You may ask, "Well, if you are so concerned about the negative impact of your work [on artificial brains] on humanity, why don't you stop it and do something else?" The truth is, I feel that I'm constructing something that may become rather godlike in future decades... The prospect of building godlike creatures fills me with a sense of religious awe that... motivates me powerfully to continue, despite the possible horrible negative consequences.
The first intelligent machine is the last invention that man need ever make since it will lead, without further human invention, to the ultraintelligent machine... To up-date Voltaire: if God does not exist we shall have constructed him, or at any rate a reasonable approximation. Or will it be the Devil?
...Before building FAI you built an oracle AI to help you. With its help, you found a mathematical definition of U, the utility of humanity’s extrapolated volition (or whatever). You were all pretty pleased with yourselves, but you didn’t stop there: you found a theory of everything, located humanity within it, and wrote down the predicate F(X) = “The humans run the program described by X.”
To top it off, with the help of your oracle AI you found the code for a “best possible AI”, call it FAI, and a proof of the theorem:
There exists a constant Best such th
If, then, men... were only machines of so complicated a make that it was less trouble to us to cut the difficulty and say that that kind of mechanism was 'being alive,' why should not machines ultimately become as complicated as we are... and to be indeed as living as it was in the nature of anything at all to be?
Samuel Butler (1880)
If you invent a breakthrough in artificial intelligence, so machines can learn, that is worth 10 Microsofts.
I am the psychotherapist. Please, describe your problems. Each time you are finished talking, type RET twice.
I feel so tired
Why do you say?
I fall asleep all the time
Is it because of your plans that you say?
Yes, I work too hard
Maybe your life has something to do with this.
Indeed!
Can you elaborate on that?
Well, I work so can improve my life
What makes you believe?
I read it somewhere
Is it because of the people you hang around with that you say?
I bet it is because of that
I would appreciate it if you would continue.
[...]
my secretary will send you a bill.
-- ema...
Members of the artificial intelligence community bear an ominous resemblance to... the Sorcerer's Apprentice. The apprentice learnt just enough magic for his master to save himself the trouble of performing an onerous task, but not quite enough to stop the spellbound buckets and brooms from flooding the castle.
Margaret Boden, Artificial Intelligence and Natural Man, p. 463
...Once a machine is designed that is good enough… it can be put to work designing an even better machine. At this point an "explosion" will clearly occur; all the problems of science and technology will be handed over to machines and it will no longer be necessary for people to work. Whether this will lead to a Utopia or to the extermination of the human race will depend on how the problem is handled by the machines. The important thing will be to give them the aim of serving human beings.
It seems probable that no mechanical brain will be really u
......it is always dangerous to try to relieve ourselves of the responsibility of understanding exactly how our wishes will be realized. Whenever we leave the choice of means to any servants we may choose then the greater the range of possible methods we leave to those servants, the more we expose ourselves to accidents and incidents. When we delegate those responsibilities, then we may not realize, before it is too late to turn back, that our goals have been misinterpreted, perhaps even maliciously. We see this in such classic tales of fate as Faust, the So
Wozniak declared to his audience that "we're already creating the superior beings, I think we lost the battle to the machines long ago."
The survival of man may depend on the early construction of an ultraintelligent machine — or the ultraintelligent machine may take over and render the human race redundant or develop another form of life. The prospect that a merely intelligent man could ever attempt to predict the impact of an ultraintelligent device is of course unlikely but the temptation to speculate seems irresistible.
Julius Lukasiewicz (1974)
A superintelligent computer designed to win at chess will keep trying to win at chess, ignoring any other goals along the way. It doesn't matter whether it's a million times smarter than Einstein, it's not going to start wanting to fight for freedom just because humans like that kind of thing any more than it's going to start wanting to have sex with pretty actors and actresses just because humans like that kind of thing. It's just going to be really, really good at winning at chess.
Our own technological development is so rapid that we must accept the fact of imminent developments which are beyond our present understanding... It is therefore certain that a society only a few hundred years more advanced than our own could, if they thought it expedient, exterminate terrestrial life without effort...
It is useless to think of any form of defence against any action by superior intelligeneces... it would be just as futile as the occupants of an antheap declaring war against a bulldozer.
Cade (1966), p. 220
Page 223 includes this drawing of self-reproducing machines.
...political leaders on Earth will slowly come to realize... that intelligent machines having superhuman thinking ability can be built. The construction of such machines, even taking into account all the latest developments in computer technology, would call for a major national effort. It is only to be expected that any nation which did put forth the financial and physical effort needed to build and programme such a machine, would also attempt to utilize it to its maximum capacity, which implies that it would be used to make major decisions of national poli
...future machines could be more intelligent than any man, and it is possible that a sort of mechanical evolution could be introduced, using existing computers to design their descendents, and so on, generation after generation, getting a little more brilliant at every step. It is in fact a theoretical possibility that, above a certain level of complexity, any computer can design a better computer than itself; this was first pointed out by the late John von Neumann.
...whenever mechanical or inorganic brains have been discussed, there have been passionate den
Assume for the sake of argument that conscious beings have existed for some 20 million years: see what strides machines have made in the last thousand! May not the world last 20 million years longer? If so, what will they not in the end become? Is it not safer to nip the mischief in the bud and to forbid them further progress?
Samuel Butler, 1872
(My own answer to Butler's question is "No" for the reason Moravec gave in 1988.)
...As the human race, we are delicately positioned. We have the... ability... to create machines that will not only be as intelligent as humans but that will go on to be far more intelligent still. This will spell the end of the human race as we know it. Is that what we want? Should we not at least have an international body monitoring and even controlling what goes on?
When the first nuclear bombs were dropped on Japan, killing thousands of people, we took stock of our actions and realised the threat that such weapons posed to our existence. Despite the resu
...A point that K. Eric Drexler makes about nanotechnology research also applies to AI research. If a capability can be gained, eventually it will be gained and we can therefore not base humanity’s survival on AI never happening. Doing so is denying the inevitable. Instead, we can only hope to manage it as well as possible. Suppose we took the view that ethical people would not create AI. By definition, the only people creating it would be unethical people, who would then control what happened next -- so by opting out, all the ethical people would be doing w
...intelligent machines will embody values, assumptions, and purposes, whether their programmers consciously intend them to or not. Thus, as computers and robots become more and more intelligent, it becomes imperative that we think carefully and explicitly about what those built-in values are. Perhaps what we need is, in fact, a theory and practice of machine ethics...
...We can already see a glimmer of how computers might make [ethical] choices in Jaime Carbonell's model of subjective understanding. Carbonell showed how programs could be governed by heirarch
Some philosophers and scholars who study and speculate on the [intelligence explosion]... maintain that this question is simply a matter of ensuring that AI is created with pro-human tendencies. If, however, we are creating an entity with greater than human intelligence that is capable of designing its own newer, better successors, why should we assume that human-friendly programming traits will not eventually fall by the wayside?
Al-Rodhan (2011), pp. 242-243, notices the stable self-modification problem.
From Michie (1982):
...Competent medical and biological research authorities in various parts of the world are concerned about genetic engineering... There is the possibility that as an accident, a side-effect of such research, some quite new and virulent micro-organism might multiply to an extent with which we are not able to cope. As a consequence, the Medical Research Council in Britain... recently supported a six-month moratorium on research in that specific area while the matter was studied more deeply and new safeguards drawn up.
It is conceivable that
When machines acquire an intelligence superior to our own, they will be impossible to keep at bay... [Human-level AI] will threaten the very existence of human life as we know it... We should not... expect the main battles of the twenty-first century to be fought over such issues as the environment, overpopulation, or poverty. No, we should expect the fight to be about how we cope with [AI]; and the issue [of] whether we or they — our silicon challengers — control the future of the earth.
Crevier, AI: The Tumultuous History of the Search for Artificial Intelligence, p. 341
"Speculations concerning the first ultraintelligent machine" started with the dubious sentence "The survival of man [sic] depends on the early construction of an ultraintelligent machine." That might have been appropriate during the Cold War. Today I suspect that the word survival should be replaced by extinction.
People tend to feel that intelligence is a good thing, even if they are unable to say exactly what it is. But its presence in a machine might not be an unmitigated blessing... there are purposes for which we might not want machines to be intelligent.
The greatest task before civilisation at present is to make machines what they ought to be, the slaves, instead of the masters of men.
Havelock Ellis, 1922
The singularity literature perhaps does a service by highlighting the ways in which AI developments could produce new degrees of intelligence and operational autonomy in AI agents—especially as current AI agents play an increasingly important role in the design of future AI agents. Bearing in mind the far-reaching implications of such possible future scenarios, the urgency of work in [machine ethics] to ensure the emergence of ‘friendly AI’ (Yudkowsky 2001, 2008) is all the more important to underline.
The establishment of contact with any superior community would obviously be of unparalleled importance for the human race — socially, scientifically, and culturally. It could lead either to our rapidly attaining superior status ourselves, or it could lead to our extinction. It probably depends upon how well we conceal, or overcome, our own grave failings as social beings.
Cade (1966), p. 228
"... This is the subject matter of Fun Theory, which ultimately determines the Fate of the Universe. For if all goes well, the question "What is fun?" shall determine the shape and pattern of a billion galaxies." - Eliezer Yudkowsky
"... This is the subject matter of Fun Theory, which ultimately determines the Fate of the Universe. For if all goes well, the question "What is fun?" shall determine the shape and pattern of a billion galaxies." - Eliezer Yudkowsky