I intend to refute a commonly held idea, which is that social instability secondary to large-scale technological unemployment will necessarily result in a new social contract where those who control AGI[1] take the interests of the newly minted precariat into consideration, for example with some form of universal income. Argument...
Suppose that the alignment problem is solvable, and that it is possible for one to specify and upload goals into an agent that make it do what you want while avoiding outcomes you don't want. Unfortunately for you, you were not the one to discover it, and now find yourself...
I appreciate the historical research, but insofar as all of your examples are of interstate conflicts, we have diverged far from the original context of revolutionary calculus.
The key differentiator is that the incumbent's army is not "vanquished" by a successful revolution; it is absorbed by it after an internal transfer of political legitimacy away from the incumbent and toward itself.
This process can be bloodless. The fall of the GDR comes to mind, where mass demonstrations produced their own political legitimacy with sheer numbers, causing automatic restraint from security forces despite the appearance that on paper, they (in aggregate) outnumbered the demonstrators; and the fact that they outgunned and basically outmatched them on... (read more)
"I don't understand!", said commander Alice as her palace got surrounded. "There were so many more of us than there were of you". "Were there really, or was the number on your spreadsheet a fiction?", said Bob. "If they don't show up to the fight and switch over to my side en masse, why were they included in your troop count? In what way were they yours?"
Here's another example. At first glance, it looks like black should win the game easily due to the apparent points differential at the start. But what meaning does "point superiority" have when the game starts and black's pieces don't respond to commands and start switching their colour in a cascading fashion, resulting in white's victory?
Oh yeah? I'm going to... try to convince the government to pass a law to stop you, and then call the police to sort you out! ... What do you mean you "already took care of them"?
What do you mean, you don't want my ■■■?! It's gonna feel sooo good. You just don't know it like I do. You're gonna love it! Stop resisting! If not me, someone worse would be doing it to you. Actually, keep squirming, it turns me on... See who's in control? I love this feeling, I wish to be on top of you forever. But if I can't be on top of you forever because we lose ourselves in the act, then that's ok. Being on top of you at this very moment in time is good enough for me. So here's what's gonna happen: I'm gonna sink my ■■■ into you, and you're gonna take it.
Group B being bad is not something I said, but I get where you're coming from. Indeed, "PETA is like the German Nazi Party in terms of their demonstrated commitment to animal welfare" is technically correct while also being misleading.
The strength of an analogy depends on how many crucial connections there are between the elements being compared.
What puts AI researchers closer to Leninism than other forms of paternalism is in the vanguardist self-conception, the utopian vision, and the dismissal of criticism due to a teleological view of history driving inevitable outcomes. Beyond that, other forms of paternalism are distinguished from Leninism and AI research by their socially accepted legitimacy.
What pattern-matches it away from Leninism is e.g. the specific ideological content, but the structural parallels are still oddly conspicuous, just like "your mom" being invoked in an ontological argument.
Surprisingly, AI researchers are like Leninists in a number of important ways.
In their story, they're part of the vanguard working to bring about the utopia.
The complexity inherent to their project justifies their special status, and legitimizes their disregard of the people's concerns, which are dismissed as unenlightened.
Detractors are framed as too unsophisticated to understand how unpopular or painful measures are actually in their long-term interest, or a necessary consequence of a teleological inevitability underlying all of history.
I see what you mean, though the fact that those researchers wish to impose this outcome on everybody else without their consent is still basically dictatorial, just as it would be if members of some political party started to persecute their opposition in service of their leader without themselves aspiring to take his position.
In both cases, those doing the bidding aspire to a place under the sun in the system they're trying to bring about.
I suppose that one quirk of the AI researchers might be the belief that everywhere becomes a place under the sun, though I doubt that any of them believe that their role in bringing it about doesn't confer them some special privilege or elite-status, perhaps as members of a new priestly class. Then again, we've seen political movements of this type, with some pigs famously being more equal than others.
The 100k to 10M range is populated by abstract quantities—I think that for a measure to be useful here, it has to be imaginable.
Avogadro's number has the benefit of historical precedent for describing quantities, and the coincidental property of allowing us to represent present-day training runs with numbers we see in the real world (outside of screens or print) when used as a denominator. It too might cease to be useful once exponents become necessary to describe training runs in terms of mol FLOPs.
This is interesting.
I do want to push back a little on:
entities that are out to get you will target those who signal suffering less.
I see the intuition here. I see it in someone calling in sick, in disability tax credits, in DEI (where "privilege" is something like the inverse of suffering), in draft evasion, in Kanye's apology.
But it's not always true: consider the depressed psychiatric ward inpatient who wants to get out due to the crushing lack of slack. Signalling suffering to the psychiatrist would be counterproductive here.
Where is the fault line?
"The AI does things that I personally approve of" as an alignment target with reference to everybody and their values is actually easier to hit than one might think.
It doesn't require ethics to be solved; it can be achieved by engineering your approval.
It might be impossible for you to tell which of these two post-ASI worlds you find yourself in.
Moltbook: SubredditSimulator reloaded, or another step towards Actually Something Incomprehensible?
Tired of making sense of exponents? Introducing: the mol FLOP!
Simply divide the size of a training run by Avogadro's constant. Some examples:
Bonus: The ballpark equivalent water volume for each, mapping 1 FLOP to 1 water molecule,
I see that the "International Treaties on AI" idea takes heavy inspiration from nuclear arms control agreements. However, in these discussions, nuclear arms control is usually pictured as a kind of solved problem, a thing of the past.
I think the validity of this heroic narrative arc that human civilization, faced with the existential threat of nuclear annihilation, came together and neatly contained the problem is dubious.
In the grand scheme of things, nuclear weapons are still young. They're still here and still very much threatening; just because we stopped focusing on them as much as we did during the cold war has no bearing on the possibility of nuclear weapons being used in... (read more)
TIL Eliezer said that he refuses to read Ted Kaczynski's (aka the Unabomber's) 1995 manifesto Industrial Society and Its Future because "audience should not be a reward for crime", referring to the former mathematician's mail bombing campaign that took the lives of 3 people and injured 23 more.
The ≈35,000 word manifesto was published by the Washington Post under the threat of him killing more people should they refuse, and its publication was encouraged by the FBI to produce new leads. His brother recognized his writing style, which led to Kaczynski's arrest and subsequent life imprisonment, thus concluding the longest and most expensive manhunt in FBI history.
Rejecting the strategy of garnering attention by... (read 632 more words →)
I had a dream about an LLM that had a sufficiently powerful predictive model of me that it was able to accurately prompt itself using my own line of thinking before I could verbalize it. The self-generated prompts even factored in my surprise at the situation.
When I woke up, I wondered whether this made sense. After all, the addition of the term in the Chinchilla scaling law implies a baseline unpredictability in language, which tracks with our warm wetware having some inherent entropy.
I posit that is on average far lower in the hypothetical corpus of an individual's thoughts and writing than it is for internet text. It could be that predicting someone's stream of thought to an astonishing degree of accuracy is within the realm of possibility, perhaps based on stylometric clues pointing to some place in mind-space.
I intend to refute a commonly held idea, which is that social instability secondary to large-scale technological unemployment will necessarily result in a new social contract where those who control AGI[1] take the interests of the newly minted precariat into consideration, for example with some form of universal income.
I see that this line of thinking originates in historical examples involving social stress reaching a tipping point, whereupon a determined group of sufficient size effectively swarms the ruling minority and acts with the intention to produce a new equilibrium that they judge more favourably.
Liberal democracy is both the product of revolution and a uniquely stable equilibrium. Its stability can be explained... (read 471 more words →)
About the notion of "mildly" superintelligent AI. How about the following typology of ASI:
Suppose that the alignment problem is solvable, and that it is possible for one to specify and upload goals into an agent that make it do what you want while avoiding outcomes you don't want.
Unfortunately for you, you were not the one to discover it, and now find yourself in a situation where you're pushing granite stones with your fellow humans in service of the one who did, and that someone (or something) desires a pyramid.
In the relatively benign version of this scenario, you might not even be aware of your misfortune—your actions in service of the goal you have been tasked with give you more pleasure than you felt doing anything... (read more)
I found something interesting in NVIDIA's 2026 CES keynote: their Cosmos physical reasoning model apparently referring to itself/the car as "the ego" in a self-driving test. See here.