... AI "taking over the world", some human or group of humans "taking over the world", whatever.

New to LessWrong?

Mentioned in
New Answer
New Comment

11 Answers sorted by

AI cognition eventually becomes more desirable than human cognition along key dimensions, including: 

  • price per cognitive output
  • maximum quality of cognitive output
  • maximum serial speed of cognition
  • creativity, morality, reliability, charisma, etc
  • ability to make meaningful and enjoyable emotional connections 
  • cross-compatibility with other cognitive systems
  • loyalty and transparency to the entity employing the cognition
  • ease with which governments can impose regulations on the cognition
  • ease of acquiring specific flavors of cognition that best meet current requirements (finding and hiring an employee with specific skills becomes prompting or finetuning a model)
  • ease of adjusting the volume of employed cognition (hiring or firing new employees becomes spinning model instances up or down)

It becomes overwhelmingly obvious that most decisions are better made by AI, and all the economic incentives point toward replacing human decision making. Eventually, AIs make the vast majority of decisions, including decisions that influence the future trajectory of civilization.

AIs no more need coercion to takeover from human cognition than text-to-image models need coercion to takeover from visual artists.

I imagine some potent mix of rapid wealth generation, unusually persuasive propaganda, weapons technology (e.g. cheap plentiful armed robots), hacking (via software and also hardware infiltration of data centers and chip fabs), unprecedentedly widespread surveillance, strategic forecasting, automated industrial production, sophisticated genetically engineered viruses, brain control technology (e.g. brain computer interfaces with input/output capability), material resource acquisition (e.g. automated asteroid mining, automated deep subterranean mining), rapid scientific R&D, and of course the many potent artificial minds powering all of this, outnumbering bio-human thoughts-per-second by multiple orders of magnitude even if they don't also have super human intelligence. And then of course, the capability gap widening rapidly since much of this has complementary positive feedback loops. Humanity is in a very fragile place right now.

What you want to happen, happens.

In case of an AI, the consequences of random quirks of your programming happen, even if people do not want them to happen.

Me, sitting on a throne, as your entirely benevolent world dictator. Oh, how did I get there? Someone posted on LessWrong and I followed their blueprint!

There are two main kinds of "take over the world" I think people have in mind:

  1. You have some level of authority over the entire world, but still have humanlike constraints. Think politics, human-scale game theory, the kind of "control" that e.g. Napoleon would've had over the French Empire at its height. Like, sure, the buck stops with him, but if he appointed his brother to King Of X and that brother was committed to a bad decision, Napoleon might not be able to really stop him. Presumably, the depth of control exercised by a leader or small group would be lower for a larger area controlled.

  2. Cartoonish totalitarian in-depth control over the entire world. Think nanobots, mind-control, near-total surveillance. The version we should be scared of with A.I. Debatably the only version that would be game-changingly useful to whichever entity or group got it.

Here is what I would do, in the hypothetical scenario, where I have taken over the world.

  1. Guard against existential risk.
  2. Make sure that every conscious being I have access to is at least comfortable as the baseline.
  3. Figure out how to safely self-modify, and become much much much ... much stronger.
  4. Deconfuse myself about what consciousness is, such that I can do something like 'maximize positive experiences and minimize negative experiences in the universe', without it going horribly wrong. I expect that 'maximize positive experiences, minimize negative experiences in the universe' very roughly points in the right direction, and I don't expect that would change after a long reflection. Or after getting a better understanding of consciousness.
  5. Optimize hard for what I think is best.

Though this is what I would do in any situation really. It is what I am doing right now. This is what I breathe for, and I won't stop until I am dead.

[EDIT 2023-03-01_17-59: I have recently realized that is is just how one part of my mind feels. The part that feels like me. However, there are tons of other parts in my mind that pull me in different directions. For example, there is one part that wants me to do lots of random improvements to my computer setup, which are fun to do, but probably not worth the effort. I have been ignoring these parts in the past, and I think that their grip on me is stronger because I did not take them into account appropriately in my plans.]

Seems like a very suitable question for ChatGPT:

To "take over the world" generally refers to the idea of gaining control or domination over all or most of the world's population, territory, resources, or political and economic systems. This could involve military conquest, political manipulation, economic coercion, or other means of exerting power or influence.

There have been many historical examples of people or groups trying to take over the world, either through direct military action or through more subtle means of manipulating and controlling societies and governments. However, the concept of taking over the world is often used more casually or metaphorically, and may refer to anything from an individual or group trying to achieve a high level of influence or control within a particular sphere of activity to someone simply having a particularly grandiose ambition or desire for power.

Basically, concentration of control, whether overt or covert (as in, toxoplasma-like) or both.

I don't know. Some of the other comments look plausible, but another picture would be that AIs do so many things so quickly that humans can't tell what is happening anymore, and thereby generally lose the ability to act.

The Human Case:

A lot of coordination work. I have a theory that humans prefer mutual information (radical, I know) so a surprising-to-other-people amount of work goes into things like implementing global holidays, a global standard educational curriculum, ensuring people get to see direct representatives of the World-Emperor during their lives at least a few times, etc. This is because shared experiences generate the most mutual information.

I feel like in order for this to succeed it has to be happening during the takeover already. I cannot give any credence to the Cobra Commander style global takeover by threatening to use a doomsday machine method, so I expect simultaneous campaigns of cultural, military, and commercial conquest to be the conditions under which these things get developed.

A corollary of promoting internal coordination is disrupting external coordination. This I imagine as pretty normal intelligence and diplomatic activity for the most part, because this is already what those organizations do. The big difference is that the true goal is taking over the world, which means the proximate goal is getting everyone to switch from coordinating externally to coordinating with the takeover. This universality implies some different priorities and a much different timescale than most conflicts - namely it allows an abnormally deep amount of shared assumptions and objectives among the people who are doing intelligence/diplomatic work. The basic strategy is to produce the biggest coordination differential possible. 

By contrast, I feel like an AI can achieve world takeover with no one (or few people) the wiser. We don't have to acknowledge our new robot overlords if they are in control of the information we consume, advise all the decisions we make, and suggest all the plans that we choose from. This is still practical control over almost all outcomes. Which of course means I will be intensely suspicious if the world is suddenly, continuously getting better across all dimensions at once, and most suspicious if high ranking people stop making terrible decisions within the span of a few years.

Follow-up question: do you know where your models/intuitions on this came from? If so, where?

(I ask because this answer comes closest so far to what I picture, and I'm curious whether you trace the source to the same place I do.)

4ryan_b1y
Yes. The dominant ones are: * Military experience: I was in the infantry for 5 years, and deployed twice. I gained an extremely high appreciation for several important things: the amount of human effort that goes into moving huge amounts of people and stuff from A to B; growth and decay in the coordination and commitment of a team; the mindblowingly-enormous gap between how a strategy looks from on high to how it looks on the ground (which mostly means looking at failure a lot). * Jayne's macroscopic prediction paper: I am extremely libertine in how I apply these insights, but the relevant intuition here is "remember the phase volume in the future" which winds up being the key for me to think about the long term in a way that can be operationalized. As an aside, this tends to break down into two heuristics - one is to do stuff that generates options sometimes, and the other is to weigh closing options negatively when choosing what to do. * Broad history reading: most germane are those times when some conqueror seized territory, and then had to hustle back two years later when it rebelled. Or those times when one of the conquering armies switched sides or struck out on their own. Or the charge of the light brigade. There are a huge number of high level coordination and alignment failures. The most similar established line to the way I think about this stuff is Boyd's OODA loop, which is also my guess for where you trace the source (did I guess right?). I confess I never actually think in terms of OODA loops. Mostly I think lower-level, which is stuff like "be sure you can act fast" and "be sure you can achieve every important type of objective."
4johnswentworth1y
Yup, exactly.

Phase transition into the unknown, or regained opportunity to properly figure things out.

The process of taking over the world, or the outcome of having taken over the world?

Outcome

  • "Absolute control" would be something like "having a veto over any action that a person or group of people deliberately takes anywhere at any scale, and being able to take any action that is knowingly available to any person or any group of people in the world, such that this capability is not diminished by your exercising it".
  • Practically, mere approximations of absolute control would do. Intuitively, the "lowest bar" to clear is something like "having enough power in a self-sustaining configuration to sustainably control global features of the world".

Process

There's two primary factors to consider, here:

  • Power is interchangeable.
    • Financial power (billions of dollars) can be exchanged for political power (bribes, lobbying), or social power (financing propaganda, advertisement, etc.), or technological power (funding/buying companies and having them develop whatever infrastructure you want).
    • Social power (being trusted/listened-to by millions of people) can be exchanged for financial or political power...
    • Technological power (having a blueprint of a technology light-years ahead of everyone else's) can be exchanged for financial power (selling it) or social power (building a personal platform on top of the innovation), or political power (monopolizing it and making a nation cater to your whims)...
    • And so on.
  • Power can beget power.
    • Money can be re-invested.
    • A politician in your pocket can let you get blackmail on other politicians, or pass laws that advantage you.
    • A large audience can be instructed to induct other people into your cult, or be made to solve math problems design technologies for you.
    • Etc.

So, what does "taking over the world" mean? It means amassing enough power in some crucial domain of society, to be able to exert global influence over every other domain of society, while always having enough "power surplus" to keep most of your power in self-perpetuating investments. (Instead of, e. g., spending it all at once on passing a single law globally, and then be left destitute. One-off global changes aren't global control.)

You can get there by controlling all the money in the world, or the most cutting-edge technologies in the world, or the most advanced communication network in the world, or by having billions of people genuinely listen to you on some important matter (religion, life philosophy...), or controlling the most powerful economy/country/military in the world, or something like this. By having control over an important resource such that the second-best alternative is hopelessly behind, so no-one would be willing to switch, and so everyone would be dependent on you.

Once you have that as a base, you can exert this global control to globally encroach on other domains, by exchanging some of your current power of whatever type you started out with into different types of power: e. g., if you started with a technological monopoly, you spread around into politics and society and finance and communications...

And as you're doing so, you're also taking care to be amassing power on net, not spending it. You crush your competition, or make sure to advance faster than them. You do that in a cross-domain fashion: you aim to set up power feedback loops in politics and technology and finance and society, and move power around to always have a "crucial stake" in every domain, and preventing any competitor to your takeover to arise in any other domain.

Eventually, you acquire enough power to just... have all the power in the world. I. e., as I'd said in "outcomes", you have a veto over any action that anyone deliberately takes at any scale, and you can cause any action that is available to any person or any group of people in the world, and this capability is not diminished by your exercising it.

And the closer you are to this ideal, along dimensions like intensity (you can introduce minor changes/moderate changes/major changes, or popular/tolerable/anti-popular changes), cross-domainity (which of the following you control: finance, technology, economy, society, etc...), fidelity (you can get precisely what you want/roughly what you want/something not unlike what you want), sustainability (how often can you cause global changes without losing power? once a decade/a year/a month/a second), the more you've "taken over the world".

(Aside: The core bottleneck here is the scalability of your control module. You can't process all that vast information if you're just a human or a small group of people, so even if you have full control, you may not know to exercise it if e. g. some sufficiently subtle threat to your regime appears, that can only be identified by cross-correlating vast quantities of information. So you'd need to solve that problem somehow. Scale your regime's ability to process information and take actions in response to it that are aligned with your values. Create truly loyal intelligent sub-commanders (while spending enormous resources on somehow opposing natural bureaucratic rot/perversity/maze dynamics)... or substitute them with AI systems, of course. Which is another reason AI takeover will be easier: digital forks provide an incontestable ability to project power and wield it with fidelity.)

2 comments, sorted by Click to highlight new comments since: Today at 11:31 AM

I turned this into a question, since that was my guess at your intention with the post.

It was, thankyou.