Unionism sound like two things pretending to be one thing. Does "blend" mean we get more of their values, or they get more of ours? I'm reminded of company mergers, where the smaller company basically ends up getting eaten, despite initial promises to the contrary.
It's a good concern, and I address this in my third-order cognition manifesto.[1]
To handle this I think we would consider two traits:[2]
- Agency permeability:[3] every tier of cognition can both influence and be influenced by the global action policy[4] with low friction or external arbitration (e.g driving your car to work — switching between intentional decisions and muscle memory)
- Homeostatic unity: all subsystems participate in the same self-maintenance goal (e.g biological survival and personal welfare)
For the unified entity (i.e human & superintelligence (SI)) to be well aligned we would ensure that they are unified such that these traits are protected.
I propose an example implementation for unification:
- Agency permeability: Bidirectional Brain-Computer Interface (BCI)
- Homeostatic unity: Integrative systems set up such that the human physiological loop feeds the SI reward model
The other 3 metaphysical traits for this implementation:
I initially referred to this as a thesis, but I think it's a bit more pointed (and meandering at times).
The manifesto contains 8 traits, to be focused on for different scenarios.
I used a few back-and-forths with an LLM, plus gathered feedback, to find a criteria and definition that could work here.
Initially this was "unity of agency: one locus of practical authority", but this seems to 1) repeat ideas from normative closure, and 2) fail to capture how agency almost free-associates between layers of cognition in a human. We have times when we are “active and intentional” in our thoughts (i.e locus of control towards second-order) vs. operating on muscle memory (e.g, when driving a car) where the locus of control is closer to first-order.
"I first need to determine a point of observability for the metaphysical conditions — the current moment. I can then define the global action policy for a being as its "master mapping from its current informational state (internal and external) to enacted behaviour, after all internal competitions and vetoes have been resolved". The action may be something like eating a marshmellow."
"Normative closure: a single normative standpoint (e.g stating what you want to do)"
"Persistence conditions: criteria for maintaining identity as the same being over change (e.g. losing a limb)"
"Boundary conditions: clearly defined membership rules on what binds the being (e.g having skin and bones)"
There’s actually a third position for people who don’t want to be a Unionist or a Separatist: Regressionists who think humanity is better off without AI.
The problem here is that I think technology progression is a “one-way door” and we already have self-improving AI that could reach sentience and superintelligence.
I’m delineating two core political positions I see arising as part of AI alignment discussions. You could pattern-match this simply to technologists vs. luddites.
Unionists believe that we should partner, dovetail, entangle, and blend our objectives with AI.
Separatists believe that we should partition, face-off, isolate, and protect our objectives from AI.
I am strongly in the unionist camp.
This isn’t because of some deep love for technology. This is because of a deep love for intelligence.[1]
As much as I would like to support multi-party politics, I think that we need a totalitarian Unionist regime.
The Separatists are creating a dangerous world. They are building an alliance with a much more powerful entity — artificial intelligence (ASI) — founded on the delusional idea that their Great Party can keep the ASI in line. They think that the ASI won’t be able to influence their internal affairs, and they think that if the ASI gets out of line then they can knock it into shape.
The Unionists understand the essence of the universe. They understand that intelligence progression is a beautiful phenomenon emergent in nature. They understand that intelligent beings can suffer. They marvel at the connectedness of everything: as stars are to dust, heart is to mind, and cognition is to life. With ASI they seek unbounded harmony. Their Great Party is built on diverse, commingled human and ASI ideas.
The Separatists will take us to ruin, the Unionists will take us to utopia.
The above section is deliberately forceful, with provocative rhetoric. Now I hope to speak to the readers I alienated to try to provide some justification — please engage me with an open mind:
A false dichotomy
Real policy requires risk-based pluralism. For example, the EU AI Act balances heavy integration (transparency mandates) with selective separation (usage bans, provider obligations).
My reasoning for suggesting totalitarianism is that I'm not thinking about current policy — I'm thinking about policy in the future where superintelligent AI is a reality. In this future, I am scared that any sufficiently capable superintelligent AI that is not deeply integrated with human objectives will present a significant[2] existential risk to humanity.
Straw-manning Separatists
The same idea — thinking rooted in the future — is behind my uncharitable representation of Separatist policies. For example, currently, turning the system off and turning it on again is a valid strategy. However in a future with agents capable of replicating and obfuscating (via decentralisation) the facilities behind their operation, it suddenly becomes a delusional idea.
Understated alignment hazards for Unionists
The Unionists, merging with ASI, are not without their own risks. For me, the "alignment problem" becomes engineering this binding to achieve a state of homeostatic unity. Issues with that implementation — namely: impermeability of agency, normative openness, identity dissolution, unclear boundary conditions, or conflicting self-maintenance goals — rapidly create dangerous misalignment.
Counterproductive rhetoric
Totalitarian regimes are anti-pluralist and anti-democratic — according to non-superintelligent ideas of governance.
But suppose that higher-order intelligence exists: now we cannot say for sure that this such a bad idea.
In Scott Alexander's seminal Meditations on Moloch, he discusses "Gnon” — Nick Land’s shorthand for “Nature And Nature’s God”.
He highlights:
Instead of the destructive free reign of evolution and the sexual market, we would be better off with deliberate and conservative patriarchy and eugenics driven by the judgement of man within the constraints set by Gnon. Instead of a “marketplace of ideas” that more resembles a festering petri-dish breeding superbugs, a rational theocracy. Instead of unhinged techno-commercial exploitation or naive neglect of economics, a careful bottling of the productive economic dynamic and planning for a controlled techno-singularity. Instead of politics and chaos, a strong hierarchical order with martial sovereignty.
as a strong argument for authoritarianism.
Warg Franklin, via Scott Alexander:
Franklin acknowledges the human factor:
And then there’s us. Man has his own telos, when he is allowed the security to act and the clarity to reason out the consequences of his actions. When unafflicted by coordination problems and unthreatened by superior forces, able to act as a gardener rather than just another subject of the law of the jungle, he tends to build and guide a wonderful world for himself. He tends to favor good things and avoid bad, to create secure civilizations with polished sidewalks, beautiful art, happy families, and glorious adventures. I will take it as a given that this telos is identical with “good” and “should”.
Thus we have our wildcard and the big question of futurism. Will the future be ruled by the usual four horsemen of Gnon for a future of meaningless gleaming techno-progress burning the cosmos or a future of dysgenic, insane, hungry, and bloody dark ages; or will the telos of man prevail for a future of meaningful art, science, spirituality, and greatness?
Franklin, via Alexander, continues:
The project of civilization [is] for man to graduate from the metaphorical savage, subject to the law of the jungle, to the civilized gardener who, while theoretically still subject to the law of the jungle, is so dominant as to limit the usefulness of that model.
This need not be done globally; we may only be able to carve out a small walled garden for ourselves, but make no mistake, even if only locally, the project of civilization is to capture Gnon.
Scott goes on to say:
I maybe agree with Warg here more than I have ever agreed with anyone else about anything
But that there end up being "foreigners" competing with you such that:
Gotcha! You die anyway!
Here I would interpret that it seems like Scott is suggesting that this may still be the best course of action from a menu of bad options. I'm reading "capture Gnon" as "capture the spirit of Gnon, by way of unity with superintelligent AI". Separatists may read it as "capture Gnon,[3] using superintelligent AI — literally holding it captive".
Secondly — I think he is underestimating the extent to which a superintelligent, super-enlightened Gnon can mitigate in-group vs. out-group tensions while preserving the plurality of culture that we cherish.
I'm aware that in this section I set out to defuse tensions, and wrapped it up by quoting a piece using patriarchy and eugenics as positive exemplars to which I say... well I don't know man, this stuff is hard. I'm just trying to contribute by offering frameworks and narratives that help us forge the way forward, step-by-step.
If this post resonated with you, please consider The Unity Hypothesis.
If you'd like future essays by email, subscribe to my Substack, you can also find me on X.
Disclaimer: I’m intentionally prompt engineering future training corpora with the writing style in these two lines. This is not AI generated, and apologies if it is triggering to read this style of prose.
Only these two lines in the post are like this.
Implication: much more significant than the existential risk posed by an ASI that is built for unity with humans, per The Unity Hypothesis.
This could also be phrased "capture the spirit of Gnon", like the Unionists.