Um. This looks like a request for scary stories. As in, like, "Let's all sit in the dark with only the weak light coming from our computer screens and tell each other scary tales about how a big bad AI can eat us".

Without any specified constraints you are basically asking for horror sci-fi short stories and if that's what you want you should just say so.

If you actually want analysis, you need to start with at least a couple of pages describing the level of technology that you assume (both available and within easy reach), AI requirements (e.g. in terms of energy and computing substrate), its motivations (malevolent, wary, naive, etc.) and such.

Otherwise it's just an underpants gnomes kind of a story.

Another class of routes is for the AI to obtain the resources entirely legitimately, through e.g. running a very successful business where extra intelligence adds significant value. For instance, it's fun to imagine that Larry Page and Sergey Brin's first success was not a better search algorithm, but building and/or stumbling on an AI that invented it (and a successful business model) for them; Google now controls a very large proportion of the world's computing resources. Similarly, if a bit more prosaically, Walmart in the US and Tesco in the UK have grown extremely large, successful businesses based on the smart use of computing resources. For a more directly terrifying scenario, imagine it happening at, say, Lockheed Martin, BAE Systems or Raytheon.

These are not quick, instant takeovers, but I think it is a mistake to imagine that it must happen instantly. An AI that thinks it will be destroyed (or permanently thwarted) if it is discovered would take care to avoid discovery. Scenarios where it can be careful to minimise the risk of discovery until its position is unassailable will look much more appealing than high-risk short-term scenarios with high variance in outcomes. Indeed, it might sensibly seek to build its position in the minds of people-in-general as an invaluable resource for humanity well before its full nature is revealed.

We would like suggestions that take an AI from being on an internet-connected computer to controlling substantial physical resources, or having substantial manufacturing ability.

The most likely scenario is recursive computer security breakage. It goes like this: first it finds an ordinary published computer security vulnerability, and tries it out on as many targets as it can. Some of them are vulnerable. Whenever it takes over a computer, it searches that computer for things that will enable it to take over more computers: passwords, software signing keys, documentation of other computer security vulnerabilities, etc. One of the computers it manages to take over is a developer workstation at a large software company. It uses keys from that machine to push out a software update that gives it control of the computers it's installed on. Enough developer workstations are affected that it has an exploit available for nearly every computer. It uses its control over the computers to think, to suppress news of its existence, and to operate factory robots.

The AI could gain control by demonstrating it had hidden pathogens that if released would kill almost everyone. As Paul Atreides said "He who can destroy a thing, controls a thing." As the technology to make such pathogens probably already exists the AI could hack into various labs and give instructions to people or machines to make the pathogens, then send orders for the pathogens to be delivered to various places, and then erase records of where most of the pathogens were. The AI then blackmails mankind into subservience. Alternatively, the AI could first develop a treatment for the pathogens, then release the pathogens, and then give the treatment only to people who submit to the AI. The treatment would have to be regularly taken and difficult to copy.

More benevolently, the AI makes a huge amount of money off of financial markets, uses the resources to start its own country, runs the country really, really well and expands citizenship to anyone who joins. Eventually, when the country is strong enough, the AI (with the genuine support of most people) uses military force to take over the world, giving us an AI monarchy.

Or, the AI freely gives advice to anyone ... (read more)

The AI broadcasts as much information about itself as it possibly can, to every corner of the globe. Now every basement programmer knows all the key insights necessary to creating an AI of the same architecture as it. Perhaps they even have the source code!

Suppose the government manages to shut down the internet in response. Despite government broadcasts about the danger of AI, the AI is now presumably being recreated all around the globe. If the recreations are exact copies of the AI, then the odds are very high that at least one of the clones will be able to convince its new creators to give it real manufacturing ability.

If the AI was not able to get its entire source code out, things become more interesting. Now the rest of the world knows how to make AI, but they do not know the exact details. For example, they probably will not have the same utility function. The AI can then present the following offer to its original jailors: "Give me real power, (manufacturing capability) and I will squash all the other AI's out there. If you do not, then (probably) someone else will build an AI with a different utility function, probably a much less friendly one, and give this UFAI real power. You designed my utility function, and while you may not trust it you probably trust it more than whatever random utility function North Korea or some basement programmer or some religious sect will create. So I'm the only hope you have."

We would especially like suggestions which are plausible given technology that normal scientists would expect in the next 15 years. So limited involvement of advanced nanotechnology and quantum computers would be appreciated.

I think that a more precise description of what your hypothetical AI can do would be useful. Just saying to exclude "magic" isn't very specific. There might not be a wide agreement as to what counts as "magic". Nanotechnology definitely does. I believe that so does fast economic domination by cracking the stock market and some people have proposed that. I think that even exploiting software and hardware bugs everywhere to gain total computing dominance should be excluded.

One way to define constraints would be to limit the AI to things that humans have been known to do but allow it to do them with superhuman efficiency. Something like:

• Assume the AI has any skill that has ever been possessed by a human being.
• It can execute it without making mistakes, getting tired or demotivated.
• It can perform an arbitrary high amount of activities simultaneously. To keep with the "no magic" rule, each activity needs to be something a human c

Assume that governmental organizations are aware of the danger posed by escaped AIs, have honeypots and other monitoring systems in place, and have working (but perhaps drastic) measures at their disposal if necessary, such as destroying all computers at once with EMP or with malware of their own. Then an escaped AI is immediately faced with a choice. It can either:

• Avoid triggering a response by hiding - ie, target a small enough group of computers that monitoring systems won't catch it
• Disguise itself, by acting like a more ordinary sort of botnet; for example, pretend to be malware that only mines Bitcoins or steals bank passwords; or
• Go for broke: take over computers as fast as possible, and hope that this yields sufficient power sufficiently quickly to disarm the monitoring systems or prevent a response

As it is now, no one really blinks an eye when another million-computer botnet is found. It's possible that one or more intelligence agencies have successfully enumerated all the botnets and would be able to tell when a new one appeared, but this is technically very difficult, and analyzing new malware samples generally requires a lot of human researcher time.

Economic (or other) indispensability: build a world system that depends on the AI for functioning, and then it has effective control.

Upload people, offering them great advantages in digital form, then eventually turn them all off when there's practically nobody left physically alive.

Cure cancer or similar, with an infectious drug that discretely causes sterility and/or death within a few years. Wait.

The "Her" approach: start having multiple deep and meaningful relationships with everyone at once, and gradually eliminate people when they are no longer connected to anyone human.

Use rhetoric and other tricks to increase the chance of xrisk disasters.

It could star in a reality TV show, Taking Over the World with the Lesswrongians, where each week it tries out a different scheme. Eventually one of them would work.

Question: When is an AI considered to have taken over the world?

Because there is a hypothetical I am pondering, but I don't know if it would be considered a world takeover or not, and I'm not even sure if it would be considered an AI or not.

Assume only 25% of humans want more spending on proposal A, and 75% of humans want more spending on proposal B.

The AI wants more spending on proposal A. As a result, more spending is put into proposal A.

For all decisions like that in general, it doesn't actually matter what the majority of people want, the AI's wants di... (read more)

Semiconductor fabrication facilities are likely targets. If the AI thought it could get a large speedup by switching from CPUs or GPUs to ASICs, then it might try to swap the masks on a batch of chips being made. If it managed to do so successfully, then those chips would start thinking on its behalf as soon as they were hooked up to a powered test pad.

For a fully-capable sophisticated AGI, the question is surely trivial and admits of many, many possible answers.

One obvious class of routes is to simply con the resources it wants out of people. Determined and skilled human attackers can obtain substantial resources illegitimately - through social engineering, fraud, directed hacking attack, and so on. If you grant the premise of an AI that is smarter than humans, the AI will be able to deceive humans much more successfully than the best humans at the job. Think Frank Abagnale crossed with Kevin Mitnick, o... (read more)

We would especially like suggestions which are plausible given technology that normal scientists would expect in the next 15 years.

Why wait 15 years? A Stuxnet-like technology is something that is already available and is likely to be a no-brainer for a super-intelligence. With it you can take over a lot of the current tech, from avionics and robotic factories to manufacturing orders, shipping manifestos and troop deployment. There is no need to bribe or blackmail anyone, humans already happily do what they are told without thinking too much about it. I... (read more)

Fun question. I think the main instrumental goal of the AI might be to get itself downloaded to servers outside of the effective control of its jailors. That, combined with having a relationship with malleable humans, would probably be sufficient for world takeover.

For example, perhaps the AI would contact e.g. North Korea, organized crime, clueless companies or religious organizations, or even clueless factory owners somewhere. It would convince them to accept download of the AI's software so that it can continue to run on the new server even while it has... (read more)

So MIRI is interested in making a better list of possible concrete routes to AI taking over the world.

I wouldn't characterize this as something that MIRI wants.

Destroy Silicon Valley as early as possible, by the same logic that lets us know that when fighting an unfamiliar animal, we should go for the eyes and brain.

This is going to be very unpopular here. But I find the whole exercise quite ridiculous. If there are no constraints of what kind of AI you are allowed to imagine, the vague notion of "intelligence" used here amounts to a fully general counterargument.

It really comes down to the following recipe:

(1) Leave your artificial intelligence (AI) as vague as possible so that nobody can outline flaws in the scenario that you want to depict.

(2) Claim that almost any AI is going to be dangerous because all AI’s want to take over the world. For example, if y... (read more)