I've also noticed this assumption. I myself don't have it, at all. My first thought has always been something like "If we actually get AGI then preventing terrible outcomes will probably require drastic actions and if anything I have less faith in the US government to take those". Which is a pretty different approach from just assuming that AGI being developed by government $x$ will automatically lead to a world with values of government $x$. But this a very uncertain take and it wouldn't surprise me if someone smart could change my mind pretty quickly.

I'm very happy this post is getting traction, because I think spotlighting and questioning these invisible assumptions should become standard practice if we want to raise the epistemic quality of AI safety discourse. Especially since these assumptions tangibly translate into agendas and real-world policy.

I must say that I find it troubling how often I see people accept the implicit narrative that “CCP AGI < USG AGI” as an obvious truth. Such a high-stakes assumption should first be made explicit, and then justified on the basis of sound epistemics. The burden of justifying these assumptions should lie on people who invoke them, and I think AI Safety discourse's epistemic quality would benefit greatly if we called out those who fail to state + justify their underlying assumptions (a virtuous cycle, I hope)

Similarly, I think its very detrimental for terms like “AGI with Western values” or “aligned with democracy” (implied positive-valences) to circulate without their authors providing operational clarity. On this note, I think it quite important that the AI Safety community isn't co-opted by their respective governments' halo terms or applause lights; let's leave it to politicians... (read more)

One very important consideration is whether they hold values that they believe are universalist, or merely locally appropriate.

For example, a Chinese AI might believe the following: "Confucianist thought is very good for Chinese people living in China. People in other countries can have their own worse philosophies, and that is fine so long as they aren't doing any harm to China, its people or its interests. Those idiots could probably do better by copying China, but frankly it might be better if they stick to their barbarian ways so that they remain too weak to pose a threat."

Now, the USA AI thinks: "Democracy is good. Not just for Americans living in America, but also for everyone living anywhere. Even if they never interact ever again with America or its allies the Taliban are still a problem that needs solving. Their ideology needs to be confronted, not just ignored and left to fester."

The sort of thing America says it stands for is much more appealing to me than a lot of what the Chinese government does. (I like the government being accountable to the people it serves - which of course entails democracy,  a free press and so on). But, my impression is that American values are held to be Universal Truths, not uniquely American idiosyncratic features, which makes the possibility of the maximally bad outcome (worldwide domination by a single power) higher.

There's also the possibility that a CCP AGI can only happen through being trained on Western data to some extent (i.e., the English language internet) because otherwise they can't scale data enough. This implies that it would probably be a "Marxism with Chinese characteristics [with American characteristics]" AI since it seems like that just raises the "alignment to CCP values" technical challenge difficulty a lot.

The state of the geopolitical board will influence how the pre-ASI chaos unfolds, and how the pre-ASI AGIs behave. Less plausibly intentions of the humans in charge might influence something about the path-dependent characteristics of ASI (by the time it takes control). But given the state of the "science" and lack of the will to be appropriately cautious and wait a few centuries before taking the leap, it seems more likely that the outcome will be randomly sampled from approximately the same distribution regardless of who sets off the intelligence explosion.

I'm relieved not to be the only one wondering about this.

I know this particular thread is granting that "AGI will be aligned with the national interest of a great power", but that assumption also seems very questionable to me. Is there another discussion somewhere of whether it's likely that AGI values cleave on the level of national interest, rather than narrower (whichever half-dozen guys are in the room during a FOOM) or broader (international internet-using public opinion) levels?

From an individual person perspective, less authoritarian ASI is better. "Authoritarian" measure here means the degree it allows itself to restrict your freedoms. 

The implicit assumption here as I understand it is that Chinese ASI would be more authoritarian than US. It may not be a correct assumption, as US has proven to commit fairly heinous things to domestic (spying on) and foreign (mass murdering) citizens.

As things stand today, if AGI is created (aligned or not) in the US, it won't be by the USG or agents of the USG. I'll be by a private or public company. Depending on the path to get there, there will be more or less USG influence of some sort. But if we're going to assume the AGI is aligned to something deliberate, I wouldn't assume AGI built in the US is aligned to the current administration, or at least significantly less so than the degree to which I'd assume AGI built in China by a Chinese company would be aligned to the current CCP. 

For more con... (read more)

USA wins on the merits of historically preferring to pretend it isn't ruling the world and mostly letting other countries do their thing, even when it has extreme military dominance (nukes)

China seems to be better at governance

On values USA is more adapted to wealth, while China has the communistic underpinnings which may be very good in a fully-automated economy.

Comes down to whether you want the easygoing less competent (and slightly psychotic) overlords or the more competent higher-strung control freaks I suppose.

I think the the assumption is that this is the USG of the last 50 years - which has flaws, but also has human rights goals and an ability to eventually change and accommodate the public’s beliefs. 

So in the scenario where AI is controlled by a strongly democratic USG, you have a much more robust “alignment” to enlightenment values and no one person with too much power. 

That said, that’s probably a flawed assumption for how the US government operates now/ over the next decade. 

I think people focus too much on "would US AGI be safer than China" and not as much on "how much safer"

In the sense that US has 15% pdoom and China has 22%, this notion that everyone needs to get onboard and help US win with full effort could be bad

Could be used (and arguably is currently being used) to be even LESS safe, and empower an authoritarian mercantilist behemoth state, and possibly invade other countries for resources 

And in general massively increase and accelerate pdoom simply on the idea that our pdoom is lower than theirs 

I mostly put this question through the same filter I do the question of Chinese vs. US hegemony/empire. China has a long history of empire and knows how to do it well. The political bureaucracy in China is well developed for preserving both itself and the empire (even within changes at the top/changes of dynasty). Culturally and socially the population seems to be well acclimated to being ruled rather than seeing government as the servant of the people (which I not quite the same as saying they are resigned to abusive totalitarianism, the empire has to be ... (read more)

Chinese culture values altruism to people who aren't family members or otherwise in the ingroup less than the West. The Western intellectual tradition has concepts like every one being equal and having human rights that don't exist the same way in Confucianism.

In practice, USG policy frequently isn't that altruistic either and takes actions based on non-altruistic reasons and later rationalizes them as being altruistic, but at least there's some ideal of altruism. Given the intellectual tradition, USG AGI might adhere to the values of the intellectual trad... (read more)

I feel the question misstates the natsec framing by jumping to the later stages of AGI and ASI. This is important because it leads to a misunderstanding of the rhetoric that convinces normal non-futurists, who aren't spending their days thinking about superintelligence.

The American natsec framing is about an effort to preserve the status quo in which the US is the hegemon. It is a conservative appeal with global reach, which works because Pax Americana has been relatively peaceful and prosperous. Anything that threatens American dominance, including giving... (read more)

Well its quite simple. If you believe that ASI is the most powerful weapon we can develop, akin to nukes, you would not want your enemy to poses that. If CCP reaches ASI before we do, they could just run it forever in reasoning analysis mode until it discovers and makes a plan to completely take down America (maybe the grid, cybersecurity, all the embedded spy technology they already have in America, or targetted misiles/drones). It could completely once and for all decapitate America in a single shot and solidify China as the undisputed global power. Chin... (read more)

In a post-scarcity world you probably want a lot of personal freedom. 

I think history is a good teacher when it comes to AI in general, especially AI we did not (at least at the time of deployment, and perhaps now, do not) fully understand. 

I too feel a temptation to imagine that a USG AGI would hypothetically have alignment with US ideals, and likewise a CCP AGI would align with CCP ideals.

That said, I struggle with, given our lack of robust knowledge of what alignment with any set of ideals would look like in an AGI system, and how we could assure them, having any certainty that these systems would align with anything... (read more)

Anglo armies have been extremely unusual historically speaking for their low rates of atrocity.

(I don't think this is super relevant for AI, but I think this is where intuitions about the superiority of the west bottoms out)