So one of the involved researchers - a bona fide world-renowned genius who had made signal contributions to the design of the computers and software involved and had the utmost credibility - made the obvious suggestion. Don’t let the arms race start. ... Instead, Nacirema should boldly deliver an ultimatum to the rival: submit to examination and verification that they were not developing the tech, or be destroyed.
Damn those politicians! Damn their laziness and greed! If only they'd had the courage to take over the world, then everything would have been fine!
Don't misunderstand; that's what's being proposed here. Hegemony would not have been enough. You need inspectors in all the research institutions, experienced in the local language language and culture. You need air inspections of every place a pile might be constructed, quite challenging in 1945. You need to do these things not just to your rival, but to everyone who aspires to become your rival. You need your allies to comply, voluntarily or not. Whenever anyone challenges your reign openly, you have to be willing and able to destroy them utterly. You can't miss even once, because when you do you won't get nuclear ...
This was pretty transparent. And I disagree with it.
I'd observe that though the peculiarities of the cold war actually made nuclear peace tougher than it would have been in most time periods, but we still made it through, but you can see that the current multi-polar world is substantially safer even though there are more countries with nuclear weapons than ever before.
Also, trying the Neumann plan (or your description of it) would have been awful, and would almost certainly have triggered conventional war if it had been followed through on; not only that but the USSR's development of the bomb did not mark the "closing of the window"; the US had more bombs and superior bomb delivery capabilities for several years after both countries had the bomb, and the US still didn't go to war. And even in retrospect that looks like the right choice, since war would have devastated Europe and eventually resulted in a "parable" essay about how the super-weapon motivated its developing nation to bloodily enforce global hegemony.
I don't think you understand what happened. Saddam thought that his former close sponsor & ally needed him against Iran because without our Sunni man in Iraq and the fear of WMDs the country would become a Shi'a & Persian pawn*. (You remember the whole Iran-Iraq War and 'exporting the revolution' and Hezbollah, right?)
Huh. How about that. Why, it looks like that's what happened under Maliki and that's why the country is currently being torn apart and the Iraqi government is inviting Iranian troops in to help restore order.
It would seem Saddam's mistake was in thinking the USA was run by rational actors, and not run by morons who would sabotage their geopolitical interests in the interests of revenge against a "guy that tried to kill my dad at one time". As my parable points out, one should not expect that sort of rational planning from the USA or indeed large countries in general.
So no, I think your objection does not hold water once one actually knows why the inspections were refused, and does not apply to the hypothetical involving Stalin.
* EDIT: BTW, I will note that this is a classic example of failing to apply the principle of charity, demonizing enemies, and...
the genius is John von Neumann
Historical note: According to Prisoner's Dilemma By William Poundstone, von Neumann didn't suggest issuing a nuclear ultimatum but instead advocated a surprise first strike against the Soviet Union. Bertrand Russell did suggest a nuclear ultimatum but with the goal of establishing a world government rather than just non-proliferation.
In a previous related discussion, I noted that I I Rabi and Enrico Fermi did propose using the threat of nuclear attack to deter the development of fusion weapons. However in my online searches, I haven't found any prominent historical figures suggesting the exact thing that you (and I in that previous thread) are suggesting here, of using a nuclear threat just to prevent the proliferation of fission weapons, which is kind of curious...
It's a classic! You might enjoy some of the other articles/parables about the Nacirema: https://en.wikipedia.org/wiki/Nacirema
Well, in reality Americans understood that Stalin would never agree to such plan, so it means war. They did not have enough nukes for guaranteed victory (few cities were acceptable losses for USSR), did not have any reliable information about Soviet nuclear research, and knew how badly war with Russia can end.
I'm not sure whether you intended this, but I suppose "70 years from now" does mean "70 years ago", in an extremely literal, unidiomatic sense of "from".
deducible by a grad student from published papers, as concerned agencies in Nacirema proved
Probably because I've read a few accounts of a grad student doing this, I realized what you were doing by this sentence.
I have to say that nuclear warfare was less of a human extinction risk than some people tend to think or is directly suggested by this text. Even a straight all out war between the United States and Soviet Union using their full arsenals would not have caused human extinction nor likely have prevented some technological societies from rebuilding if they didn't outright survive. I've seen out there expert analyses on raw destruction and on factors like subsequent global climate devastation showing this conclusion from any plausible military contigencies a...
The analogy seems pretty nice. The argument seems to be that, based on the historical record, we're doomed to collective inaction in the face of even extraordinarily dangerous risks. I agree that the case of nukes does provide some evidence for this.
I think you paint things a little too grimly, though. We have done at least a little bit to try to mitigate the risks of this particular technology: there are ongoing efforts to prevent proliferation of nuclear weapons and reduce nuclear stockpiles. And maybe a greater risk really would provoke a more serious response.
Your post prompted me to recall what I read in Military Nanotechnology: Potential Applications and Preventive Arms Control by Jürgen Altmann. It deals mostly with non-molecular nanotech we can expect to see in the next 5-20 years (or already, as it was published in 2006), but it does go over molecular nanotech and it's worth thinking about the commonly mentioned x-risk of a universal molecular assembler in addition to AGI for the elites to handle over the next 70 years.
I think as a small counter to the pessimistic outlook the parable gives, it's worth reme...
every sector benefits from electricity ‘too cheap to meter’
Except that that was never actually in the cards...
Why, did you think it was about something else?
I patternmatched the first half to eugenics.
Well, "impoverished foreign country" doesn't match well to Nazi Germany, but everything else checks out.
Good story! The bit about Nacirema's rival being 'to the east' and an 'orietal despotism' and 'angry at historical interference in its domestic affairs by Seilla & Nacirema' gave the game away a little early though.
Let me tell you a parable of the future. Let’s say, 70 years from now, in a large Western country we’ll call Nacirema.
One day far from now: scientific development has continued apace, and a large government project (with, unsurprisingly, a lot of military funding) has taken the scattered pieces of cutting-edge research and put them together into a single awesome technology, which could revolutionize (or at least, vastly improve) all sectors of the economy. Leading thinkers had long forecast that this area of science’s mysteries would eventually yield to progress, despite theoretical confusion and perhaps-disappointing initial results and the scorn of more conservative types and the incomprehension (or outright disgust, for ‘playing god’) of the general population, and at last - it had! The future was bright.
Unfortunately, it was hurriedly decided to use an early prototype outside the lab in an impoverished foreign country. Whether out of arrogance, bureaucratic inertia, overconfidence on the part of the involved researchers, condescending racism, the need to justify the billions of grant-dollars that cumulative went into the project over the years by showing some use of it - whatever, the reasons no longer mattered after the final order was signed. The technology was used, but the consequences turned out to be horrific: over a brief period of what seemed like mere days, entire cities collapsed and scores - hundreds - of thousands of people died. (Modern economies are extremely interdependent and fragile, and small disruptions can have large consequences; more people died in the chaos of the evacuation of the areas around Fukushima than will die of the radiation.)
An unmitigated disaster. Worse, the technology didn’t even accomplish the assigned goal - that was thanks to a third party’s actions! Ironic. But that’s how life goes: ‘Man Proposes, God Disposes’.
So, what to do with the tech? The positive potential was still there, but no one could doubt anymore that there was a horrific dark side: they had just seen what it could do if misused, even if the authorities (as usual) were spinning the events as furiously as possible to avoid frightening the public. You could put it under heavy government control, and they did.
But what was to stop Nacirema’s rivals from copying the technology and using it domestically or as a weapon against Nacirema? In particular, Nacirema’s enormous furiously-industrializing rival far to the East in Asia, which aspired to regional hegemony, had a long history of being an “oriental despotism” and still had a repressive political system - ruled by an opaque corrupt oligarchy - which abrogated basic human rights such as free speech, and was not a little racist/xenophobic & angry at historical interference in its domestic affairs by Seilla & Nacirema…
The ‘arms race’ was obvious to anyone who thought about the issue. You had to obtain your own tech or be left in the dust. But an arms race was terrifyingly dangerous - one power with the tech was bad enough, but if there were two holders? A dozen? There was no reason to expect all the wishes to be benign once everyone had their own genie-in-a-bottle. It would not be hyperbolic to say that the fate of global civilization was at stake (even if there were survivors off-planet or in Hanson-style ‘disaster refuges’, they could hardly rebuild civilization on their own; not to mention that a lot of resources like hydrocarbons have already been depleted beyond the ability of a small primitive group to exploit) or maybe even the human race itself. If ever an x-risk was a clear and present danger, this was it.
Fortunately, the ‘hard take-off’ scenario did not come to pass, as each time it took years to double the power of the tech; nor was it something you could make in your bedroom, even if you knew the key insights (deducible by a grad student from published papers, as concerned agencies in Nacirema proved). Rather, the experts forecast a slower take-off, on a more human time-scale, where the technology escalated in power over the next two or three decades; importantly, they thought that the Eastern rival’s scientists would not be able to clone the technology for another decade or perhaps longer.
So one of the involved researchers - a bona fide world-renowned genius who had made signal contributions to the design of the computers and software involved and had the utmost credibility - made the obvious suggestion. Don’t let the arms race start. Don’t expose humanity to an unstable equilibrium of the sort which has collapsed many times in human history. Instead, Nacirema should boldly deliver an ultimatum to the rival: submit to examination and verification that they were not developing the tech, or be destroyed. Stop the contagion from spreading and root out the x-risk. Research in the area would be proscribed, as almost all of it was inherently dual-use.
Others disagreed, of course, with many alternative proposals: perhaps researchers could be trusted to self-regulate; or, related research could be regulated by a special UN agency; or the tech could be distributed to all major countries to reach an equilibrium immediately; or, treaties could be signed; or Nacirema could voluntarily abandon the technology, continue to do things the old-fashioned way, and lead by moral authority.
You might think that the politicians would do something, even if they ignored the genius: the prognostications of a few obscure researchers and of short stories published in science fiction had turned out to be truth; the dangers had been realized in practice, and there was no uncertainty about what a war with the tech would entail; the logic of the arms race has been well-documented by many instances to lead to instability and propel countries into war (consider the battleship arms race leading up to WWI); the proposer had impeccable credentials and deep domain-specific expertise and was far from alone in being deeply concerned about the issue; there were multiple years to cope with the crisis after fair warning had been given, so there was enough time; and so on. If the Nacireman political system were to ever be willing to take major action to prevent an x-risk, this would seem to be the ideal scenario. So did they?
Let's step back a bit. One might have faith in the political elites of this country. Surely given the years of warning as the tech became more sophisticated, people would see that this time really was different, this time it was the gravest threat humanity had faced, that the warnings of elite scientists of doomsday would be taken seriously; surely everyone would see the truth of proposition X, leading them to endorse Y and agree with the ‘extremists’ about policy decision Z (to condense our hopes into one formula); how can we doubt that policy-makers and research funders would begin to respond to the tech safety challenge? After all, we can point to some other instances where policymakers reached good outcomes for minor problems like CFC damages to the atmosphere.
So with all that in mind, in our little future world, did the Nacireman political system respond effectively?
I’m a bit cynical, so let’s say the answer was… No. Of course not. They did not follow his plan.
And it's not that they found a better plan, either. (Let's face it, any plan calling for more war has to be considered a last resort, even if you have a special new tech to help, and is likely to fail.) Nothing meaningful was done. "Man plans, God laughs." The trajectory of events was indistinguishable from bureaucratic inertia, self-serving behavior by various groups, and was the usual story. After all, what was in it for the politicians? Did such a strategy swell any corporation’s profits? Or offer scope for further taxation & regulation? Or could it be used to appeal to anyone’s emotion-driven ethics by playing on disgust or purity or in-group loyalty? The strategy had no constituency except those who were concerned by an abstract threat in the future (perhaps, as their opponents insinuated, they were neurotic ‘hawks’ hellbent on war). Besides, the Nacireman people were exhausted from long years of war in multiple foreign countries and a large domestic depression whose scars remained. Time passed.
Eventually the experts turned out to be wrong but in the worst possible way: the rival took half the time projected to develop their own tech, and the window of opportunity snapped shut. The arms race had begun, and humanity would tremble in fear, as it wondered if it would live out the century or the unthinkable happen.
Good luck, you people of the future! I wish you all the best, although I can’t be optimistic; if you survive, it will be by the skin of your teeth, and I suspect that due to hindsight bias and near-miss bias, you won’t even be able to appreciate how dire the situation was afterwards and will forget your peril or minimize the danger or reason that the tech couldn’t have been that dangerous since you survived - which would be a sad & pathetic coda indeed.
The End.
(Oh, I’m sorry. Did I write “70 years from now”? I meant: “70 years ago”. The technology is, of course, nuclear fission which had many potential applications in civilian economy - if nothing else, every sector benefits from electricity ‘too cheap to meter’; Nacirema is America & the eastern rival is Russia; the genius is John von Neumann, the SF stories were by Heinlein & Cartmill among others - the latter giving rise to the Astounding incident; and we all know how the Cold War led civilization to the brink of thermonuclear war. Why, did you think it was about something else?)
This was written for a planned essay on why computational complexity/diminishing returns doesn't imply AI will be safe, but who knows when I'll finish that, so I thought I'd post it separately.