Crosspost from my blog.

Regime change

Conjecture: when there is regime change, the default outcome is for a faction to take over—whichever faction is best prepared to seize power by force.

One example: The Iranian Revolution of 1978-1979. In the years leading up to the revolution, there was turmoil and broad hostility towards the Shah, across many sectors of the population. These hostilities ultimately combined in an escalation of protest, crack-down, more protest from more sectors (protests, worker strikes). Finally, the popular support for Khomeini as the flag-bearer of the broad-based revolution was enough to get the armed forces to defect, ending the Shah's rule.

From the Britannica article on the aftermath:

On April 1, following overwhelming support in a national referendum, Khomeini declared Iran an Islamic republic. Elements within the clergy promptly moved to exclude their former left-wing, nationalist, and intellectual allies from any positions of power in the new regime, and a return to conservative social values was enforced. The Family Protection Act (1967; significantly amended in 1975), which provided further guarantees and rights to women in marriage, was declared void, and mosque-based revolutionary bands known as komītehs (Persian: “committees”) patrolled the streets enforcing Islamic codes of dress and behaviour and dispatching impromptu justice to perceived enemies of the revolution. Throughout most of 1979 the Revolutionary Guards—then an informal religious militia formed by Khomeini to forestall another CIA-backed coup as in the days of Mosaddegh—engaged in similar activity, aimed at intimidating and repressing political groups not under the control of the ruling Revolutionary Council and its sister Islamic Republican Party, both clerical organizations loyal to Khomeini. The violence and brutality often exceeded that which had taken place under the shah.

(What resulted in the following decades, was a brutally repressive and regionally violently corrosive theocratic regime.)

So we have a trajectory that goes like this:

• There is instability in, dissatisfaction with, and hostility towards the current regime.
• There are several factions with different values, beliefs, and goals, but who share an aim to topple the regime.
• Things come to a head, and the regime falls.
• One of the factions takes over. This faction doesn't especially have to be good, sane, or desired by the people. It just has to be adept and prepared to seize power.

I'm probably inaccurately oversimplifying the Iranian revolution, because I don't know the history. So this is only a conjecture. Other possible examples:

• The 1917 revolution in Russia, followed by the Bolsheviks seizing power.
• The Weimar Republic being weak, allowing the Nazis to use violence and dirty politics to dismantle democracy.
• The collapse of Yugoslavia led to e.g. Slobodan Milošević's authoritarian rule in Serbia and Montenegro.

(I'd be interested in reading a good treatment of this conjecture.)

The LLM revolution

Large language models where a shock to almost everyone's anticipations. We didn't expect to have AI systems that can talk, do math, program, read, etc. (Or at least, do versions of those activities that are only distinguishable from the real versions if you pay close attention.)

There are two common reactions to this shock:

• Deny that LLMs do much of anything that's impressive or useful.
• Claim that AGI has been achieved or will soon be achieved, as we have witnessed outputs from a generally intelligent system.

The first reaction is to deny that there's something that demands a large update. The second reaction is to make a specific update: We see generally intelligent output, so we update that we have AGI. I have argued that there should have been, inter alia, another update:

There is a missing update. We see impressive behavior by LLMs. We rightly update that we've invented a surprisingly generally intelligent thing. But we should also update that this behavior surprisingly turns out to not require as much general intelligence as we thought.

It's pretty weird that LLMs can do what they can do, but so far haven't done anything that's interesting and superhuman and general. We didn't expect that beforehand. Our previous hypotheses are not good.

We should have been trying hard to retrospectively construct new explanations that would have predicted the observations. Instead we went with the best PREEXISTING explanation that we already had. Since "nothing to see here" is, comparatively, a shittier explanation than "AGI ACHIEVED", we go with the latter. Since all our previous hypotheses were not good, we become confident in not-good hypotheses.

Finally, we have the seizing of power. Due to deference and a desire to live in a shared world, the hypothesis that survived the culling takes over.

Some readers will be thinking of Kuhn. But in Kuhn's story, the new paradigm is supposed to better explain things. It's supposed to explain both the old phenomena and also the anomalies that busted the old paradigm.

Here, instead, we have a power vacuum. There are no good explanations, no good alternative paradigms. We have a violent revolution, not a scientific one, in which the hypotheses that get promoted are those whose adherents were best prepared to seize mindshare.