I'd prefer a list that includes not just arguments for but relevant arguments against. That's better not just for rationality, but in general simply as a matter of rhetoric, lists that are deliberately one sided don't look nearly as persuasive.
In particular, I'd expand "Improvements of algorithms can in many cases lead to dramatic performance gains." to have the following counterarguments:
1) Many important problems such as linear programming and finding GCDs have close to optimal algorithms already.
(Status: uncontroversial)
2) If the complexity hierarchy doesn't collapse then many practical problems are intrinsically difficult.
(Status:uncontroversial)
2a) The hierarchy probably does not collapse.
(Status: uncontroversial for P and NP. Among experts, it is considered likely that P, NP, co-NP, EXP, and PSPACE are all distinct.)
Counterargument to 2/2a:
Basic complexity classes only look at worse case scenarios. The vast majority of instances of "hard"classes of problems are in fact easy.
(Status:Uncontroversial)
I'd prefer a list that includes not just arguments for but relevant arguments against. That's better not just for rationality, but in general simply as a matter of rhetoric, lists that are deliberately one sided don't look nearly as persuasive.
More on the Light Side of things, you want to argue with contrary positions that are probably held by the target audience.
To XiXiDu: please stop deleting your comments, it distorts the flow of discussion. Take full ownership of your mistakes, it won't actually hurt you here, and associated emotions could drive you more to improve in the future.
Take full ownership of your mistakes, it won't actually hurt you here...
You should know that I am doing that quite often. As I said in the other reply, I deleted them shortly after I posted them when I noticed that they didn't add anything valuable. I saw no replies when I deleted them. It was not my intention to disrupt the discussion.
I would find them useful if they came with an EDIT: appendix explaining how they were worthless and how you came to realize their lack of worth; as Quirrel says, learn to lose.
You should know that I am doing that quite often.
I disapprove of the other instances for the same reasons.
The algorithms that we use today for speech recognition, for natural language translation, for chess playing, for logistics planning, have evolved remarkably in the past decade. It’s difficult to quantify the improvement, though, because it is as much in the realm of quality as of execution time.
So... speech recognition has plateaued for the last decade. (Quality is easily quantifiable by error rate.) I don't know about the others, though I hear there is improvement in Go-playing algorithms.
For speech recognition recent progress has varied by problem. Speech recognition on a conversation with multiple speakers and background noise has not made good progress recently, but restricted conditions (e.g. one speaker doing voice dictation or interacting with a computational agent) have shown good progress, e.g. Dragon NaturallySpeaking.
Claim: Morality is fragile and not imperative (i.e. is not a natural law)
Status: Uncontroversial.
This sense of "uncontroversial" is not helpful if you are talking to people outside LW, and that's your stated goal. This is a very much controversial question (as well as some others that you've termed this way). Convincing people to take AI risks seriously is hard, and anything less than LW sequences isn't known to work anywhere as reliably, so it won't be a small feat to create a short summary that does work, and claiming that many controversial claims are actually "uncontroversial" won't help in this task.
This sense of "uncontroversial" is not helpful if you are talking to people outside LW, and that's your stated goal.
It wasn't my intention to tell people that the arguments are uncontroversial, that was meant to support the discussion here. But now that you mention it, I believe it could actually make people take a second look. "Wait, they say that argument is uncontroversial? Interesting! Where does that belief come from?"
This is a very much controversial question.
It was also not my intention that one should talk to religious nutters about this topic. And I don't think it is controversial for anyone else. If humans can kill humans, machines can do so more effectively. That's pretty much self-evident.
But now that you mention it, I believe it could actually make people take a second look. "Wait, they say that argument is uncontroversial? Interesting! Where does that belief come from?"
This is a very weak argument. I could as well imagine the other reply: "They say that conclusion is uncontroversial? But this statement is false, I know that many people dispute that statement. They must be presenting a sloppy and one-sided argument, I won't waste my time reading further."
Claim: Morality is fragile and not imperative (i.e. is not a natural law)
It was also not my intention that one should talk to religious nutters about this topic. And I don't think it is controversial for anyone else.
Unfortunately, it is.
Yes, The Singularity is the Biggest Threat to Humanity Saturday
Saturday, and every other day as well.
An amusing copy/paste error that you probably ought to fix.
Regarding, footnote 1, two more relevant examples: First, Simon Colton has made a series of computer programs which can construct new definitions in number theory and make conjectures about them. See this paper. Second, the Robbins conjecture, a long time open problem in abstract algebra, was proven with some human intervention but mainly an automated theorem prover.
In that case I suggest changing it from Anissimov's version to "terminal goals". Because "instrumental goals" in that context is at the very best misleading.
Claim: Algorithmic intelligence can be creative and inventive.
What do you mean, "algorithmic"? The reference you give is PR nonsense of little substance and doesn't demonstrate creativity in any relevant sense. That intelligence can be sufficiently creative is demonstrated by humans.
The reference you give is PR nonsense...
The abstract of scientific papers as PR nonsense? If such studies are not enough, what else do you suggest is valuable evidence?
...and doesn't demonstrate creativity in any relevant sense.
If composing music, deriving scientific laws and generating functional hypotheses doesn't count, what else?
That intelligence can be sufficiently creative is demonstrated by humans.
This kind of creativity may arise from our imperfect nature. It is important to demonstrate that this can be formalized, intelligently designed.
The abstract of scientific papers as PR nonsense?
What wedrifid said. In this particular case, the interpretation of the results as supporting possibility of "creativity" in any interesting sense is nonsense, even if the results themselves are genuine. Also, in the vast majority of cases, abstracts are adequate to tell you whether you want to read the paper, not to communicate the (intended meaning of) results.
The abstract of scientific papers as PR nonsense?
Frequently. Most notably in medical science (where the most money is involved) but to a lesser extent elsewhere. This particular instance is borderline. I would have to read the rest of the paper to judge just how far the spin has taken the description.
If composing music, deriving scientific laws and generating functional hypotheses doesn't count, what else?
Actually doing those things.
It is, of course, impressive that the researchers were able to solve the problems in question in a particularly general way.
To my knowledge so far nobody has put the evidence, various arguments and indications together in one place.
A relevant link to SIAI short summary of a case for AI risk:
An easy to read list of evidence and simple arguments in support of risks from artificial intelligence could help to raise awareness. Such a list could be the first step to draw attention, to spark interest and make people read some more advanced papers or the sequences. To my knowledge so far nobody has put the evidence, various arguments and indications together in one place.
My intention is to enable people interested to mitigate risks from AI to be able to offer a brochure that allows them to raise awareness without having to spend a lot of time explaining the details or tell people to read through hundreds of posts of marginal importance. There has to be some promotional literature that provides a summary of the big picture and some strong arguments for action. Such a brochure has to be simple and concise enough to arouse interest in the reader even if they just skim over the text.
Post a comment with the the best argument(s) for risks from AI
Some rules:
For starters I wrote a quick draft below. But there sure do exist a lot of other arguments and indications for why risks from artificial intelligence should be taken serious. What convinced you?
Claim: Creation of general intelligence is possible.
Status: Uncontroversial
Claim: Intelligence can be destructive.
Status: Uncontroversial
Claim: Algorithmic intelligence can be creative and inventive.
Status: Uncontroversial1
Claim: Improvements of algorithms can in many cases lead to dramatic performance gains.
Status: Uncontroversial2
Claim: Human-level intelligence is not the maximum.
Status: Very likely3
Claim: Any sufficiently advanced AI will do everything to continue to keep pursuing terminal goals indefinitely.
Status: Controversial but a possibility to be taken seriously.4 We don't yet have a good understanding of intelligence but given all that we know there are no good reasons to rule out this possibility. Overconfidence can have fatal consequences in this case.5
Claim: Morality is fragile and not imperative (i.e. is not a natural law)
Status: Uncontroversial.6 Even humans who have been honed by social evolution to consider the well-being of other agents can overcome their instincts and commit large scale atrocities in favor of various peculiar instrumental goals.
1.
— The Automation of Science
— Computer Program Self-Discovers Laws of Physics
— Genetic Algorithms and Evolutionary Computation
— Triumph of the Cyborg Composer
2.
— Page 71, Progress in Algorithms Beats Moore’s Law (Report to the President and Congress: Designing a Digital Future: Federally FUnded R&D in Networking and IT)
3.
The following argument is not directly applicable but can similarly be made for human intelligence, computational capacity or processing speed.
— Disjunctions, Antipredictions, Etc.
4.
— Basic AI Drives, Yes, The Singularity is the Biggest Threat to Humanity
5.
— What should a reasonable person believe about the Singularity?
6.
— Terry Pratchett, Unseen Academicals