[Cross posted on windowsontheory; see here for my prior writings ; update 8/25/23: added a paragraph on secrecy]

 [it appears almost certain that in the immediate future, it would be] possible to set up a nuclear chain reaction in a large mass of uranium by which vast amounts of power and large quantities of new radium-like elements would be generated.
– Letter from Albert Einstein (prepared by Leo Szilard) to F.D. Roosevelt, August 1939  

Do you know, Josef Vassarionovich, what main argument has been advanced against uranium? “It would be too good if the problem could be solved. Nature seldom proves favorable to man.” — Letter from Georgi Flerov to Joseph Stalin, April 1942. 

I’ve heard great things about Richard Rhodes’ “The Making of the Atomic Bomb.” Finally, on vacation, I managed to read it. (Pro-tip: buy the Kindle version - the hard copy is far too big to lug around.) It’s as great as people say. Can’t recommend it enough. I can’t remember when, if ever, I’ve read a book that combines so well popular science and history. Indeed, the Atomic bomb is the one setting where the precise details of the smallest particles have profoundly impacted human history. 

Here are some quick thoughts after reading the book. (Warning: spoilers below for people who don’t know how WWII ended.) 

The level of investment in the Manhattan Project was truly staggering.

I knew it but didn’t fully grasp this. This is not just the numbers ($2B, which was almost 1 percent of GDP at the time) but also the project's sheer size, employing more than 100,000 people, and the massive construction of buildings, factories, and roads at multiple sites. As just one example, when they didn’t have enough copper, the treasury department lent the project 15,000 tons of silver to be used in the electromagnetic separation plant (to be later melted and returned after the war).
 

Much of this cost was due to the compressed schedule.

The staggering cost was mainly due to the need to get the bomb done in time to use in the war. Time and again, whenever the project faced a choice between approaches A, B, or C, they chose to pursue all three in parallel, so if two failed, they could still go ahead. Whenever there was a choice between saving money or time, they opted for the latter. The fact that the cost was primarily due to time is also evidenced by the fact that, following the war, many countries could set up their own atomic bomb programs or reach the threshold of doing so at a much lower cost.

This seems to be a general principle in technological innovation: the cost of achieving a new advance decreases exponentially in time. Thus, achieving X transitions over time from being impossible to being inevitable. This is related to Bill Gates’ famous quote that in technology, we tend to overestimate progress in two years and underestimate progress in ten years.
 

The Manhattan Project was trying to achieve the Atomic bomb just at the cusp of it being possible. The project got going when General Groves was appointed (September 1942), and it took a little less than three years until the successful test (July 1945). Of course, they could have started much earlier: Einstein and Szilard sent their famous letter to Roosevelt in August 1939. The “impossible vs. inevitable” phenomenon is manifested in another way. The U.S. drastically underestimated how long it would take for the Soviet Union to achieve the bomb (even considering the Soviet advantages due to spying, which the Americans should at least have partially anticipated as well).  

The government fully trusted the scientists on the science.

The project was authorized primarily based on pen and paper calculations. At the time the project was approved, no chain reaction had been demonstrated, and the total quantity of Uranium 235 and Plutonium was minuscule, with no proof that they could be separated at scale. The U.S. trusted the scientists when they said it was possible, even though they were also very honest about the uncertainties and limitations. (The government didn’t trust the same scientists on their politics and allegiances and continually spied on the very people that it placed at the helm of its most secret and expensive venture.)
 

Many scientists independently realized the possibility of an atomic bomb.

I didn’t know how many different scientists in Germany, France, Russia, Japan, Britain, and the U.S. all realized the possibility of a fission bomb once Otto Hahn and  Fritz Strassmann observed fission experimentally in 1938, and it was analyzed theoretically by Lise Meitner and Otto Frisch. It truly was inevitable.
 

The project was secret to the public but public to the US’ adversaries.

While the Manhattan Project was highly classified, physicists in many countries, including Germany, Russia, and Japan, noticed the sudden disappearance of their American colleagues from the physics literature and deduced that the U.S. had a major Atomic fission project. The Russians, in particular, had detailed information on the project from several spies, chief among them Klaus Fuchs. That said, despite Stalin knowing about the project, and authorizing a Soviet project already in 1942, seeing the Bomb used in Hiroshima had a profound effect on him and caused him to massively increase resources and intensity.

Several scientists aware of the project, and in particular Niels Bohr, pushed on the one hand for increased openness in sharing general information about Atomic research between countries, and on the other hand for avoiding using the weapon except as a last resort. In retrospect, they were right on both counts. Secrecy on high-level information did not prevent the arms race, but the psychological effect of using the bomb spurred it on. More specific details, for example, the phenomena of “reactor poisioning” and the implosion mechanism, were very useful for Russia and by Rhodes’ estimates, probably sped up their project by two years. However, despite all the secrecy measures, including tailing Oppenheimer and other top scientists, US intelligence failed to realize the extensive spying that was going on under their noses.

Following World War II, secrecy probably had the effect of accelerating the arms race. Over-inflated estimates of Soviet capabilities were used by those who successfully pushed for the U.S. to undertake an “all out” program to develop the Hydrogen bomb.

How long was the road from theoretical possibility to actually building the bomb.

Saying the bomb was “inevitable” doesn’t mean that it was all just “engineering details.” The book details the many unexpected obstacles that the project encountered and the brilliant ideas they needed to overcome them. Just as a fissionable material can undergo a chain reaction once it is dense enough beyond its “critical mass,” so was the Manhattan Project’s success based on creating a density of talent that enabled a “critical mass” of exchanging ideas. 

How terrible of a weapon the bomb is.

It is easy to be numbed by fatality numbers. Still, the book’s descriptions of eyewitness accounts from Hiroshima bring out the sheer destructiveness of the bomb, both immediately and in the aftermath. It is purely a device of death.
 

How little sense the H bomb makes.

Reading the accounts of the destructiveness at Hiroshima, the last question that comes to mind is whether we could build a bomb that is 100 or 1000 times more powerful. And yet, sadly, this is what humanity chose to do. This also brings home the point that, like it or not, apparently any war-making device that can be built will be built, even if its only possible use is the self-destruction of our race. To understand better how we’ve reached this point, I’m now reading Rhodes’ other book, “Dark Sun.”
 

Are there lessons for AI?

We are currently in another time of great scientific advances that promise to change human society. What are the similarities and differences between the Atomic bomb and Artificial Intelligence?

AI is not the bomb 1: AI is a dual-use technology
 

The uncontrolled release of vast quantities of energy cannot serve any purpose other than destruction. Thus the atomic bomb is an instrument of death. The more powerful the bomb is, the more dangerous it is. (Nuclear fission generally is, of course, a dual-use technology.) In contrast, AI is a general technology that can be used for many purposes. In fact, it’s hard to think of a goal that AI cannot benefit. This implies a fundamental difference in the risk profile of the bomb and AI. Unlike what is sometimes suggested, AI safety and capabilities are not fundamentally opposed to one another. In some cases, such as self-driving cars, it is clear that making AI systems more capable also means making them safer. In other settings, the relation is more complicated, where AI capabilities improvements help both the “attacker” and “defender,” and it is unclear which side is helped the most:

• In cyber warfare, hackers can use AI systems to find vulnerabilities and hack software systems. But, by the same token, software vendors can use AI to patch vulnerabilities in advance before they are discovered and exploited. It is unclear which side is intrinsically more favored (assuming they both have access to AIs of equal strength). One major advantage AI offers for the defender side is coverage and timeliness. Currently, software vendors can spare limited effort toward securing systems, and this effort is often proportional to the amount that the system or individual component is used. Thus often, it is the more esoteric features (e.g., TLS heartbeat extension)  that lack attention from developers and can have security vulnerabilities survive for an extended time. 
• A similar dynamic exists in biowarfare: AI can be used to design bioweapons, but also to prepare in advance vaccines, as well as accelerate vaccine and therapeutic developments for both natural and designed pandemics.
• Disinformation is another setting in which AI can help both those who create or spread disinformation as well as those who try to prevent it. AI can make “deep fakes” and can also create highly personalized misleading messages. However, AI can also be used by social media companies to monitor misleading messages, whether on servers or on the client’s device. Hence, AI can detect deepfakes (whether by watermarking or using cryptography to establish a “chain of custody” and authenticity of the media) and warn users of influence attempts. Once again, it is unclear which side will be favored, and regulations that ensure the right set of incentives might play a considerable role. 

Eliezer Yudkowsky likes to say that “every eighteen months, the minimum IQ necessary to destroy the world drops by one point.” I believe this quote is wrong on several levels. First, it fetishizes intelligence. People have been destroying large parts of the world since the days of Genghis Khan. The Manhattan Project did indeed require the work of highly intelligent scientists, but they were only one component of the project and were successful not just because of their intelligence. (One passage in Rhodes’ book describes how Oppenheimer was so successful as a lab director also because he was attuned to the emotional needs of the scientists, with one example being him taking the time for weekly meetings with Teller.) As the infamous Oppenheimer-Truman meeting showed (not to mention the actions of Hitler and Stalin), ultimately, it was not the most intelligent people responsible for the most destruction. Second, as I discuss above, computational capabilities have a dual impact, and it is unclear, to say the least, if technological improvements make the world safer or more dangerous. 

AI is not the bomb 2: It is much more complex.

One of the striking facts in the book is the contrast between how hard it was to build the atomic bomb and how (reasonably) easy it was to predict it in advance and get order-of-magnitude estimates of its capabilities. Multiple physicists made pen and paper calculations of the critical mass of U235 and the size of a bomb.  Before Plutonium was ever created, physicists could roughly predict its potential for fission. In contrast, our ability to predict the future capabilities of AI systems is very restricted. (See this post for my crude attempts at back-of-envelope calculations, sticking with qualitative predictions only.)  We can predict near-term advances on particular benchmarks fairly reliably. But extending these predictions further out or trying to map out what these benchmarks will mean for real-world impact is still highly problematic. One worrying trend is that people sometimes replace principled analysis (as we had with fission) with opinion polls of scientists or other dubious calculations, which can give a false impression of quantifiability (“there is X% likelihood of outcome Y”) but are really “cargo cult science.” 
 

AI is like the bomb: the impossible vs. inevitable phenomenon.

Like the case of the bomb, I think building AI systems, including AGI, will become significantly easier with time, and the plot above will also hold for AI. Hardware, engineering, and algorithmic insights will make AIs more capable and cheaper to build and deploy. In particular, I believe that we will discover more efficient ways to train AI systems than the current “brute force” approach of using O(N²) operations to train an N-sized model on O(N) pieces of data; since each data point, on average, contributes O(1) bits of information to the model, it should not require Ω(N) time to process it.

Being the first mover will require significant expenses but might also confer significant advantages. This does mean that probably any type of “pauses” or “delays” are unlikely to make a long-term difference but may create more of an “overhang” effect, where multiple parties (companies/countries) can achieve similar capabilities at about the same time. For example, if the Manhattan Project didn’t exist, the Atomic bomb would have been delayed by several years, but when it was built, it would have likely been done by multiple countries.  Whether such a “multipolar” scenario is good or bad is hard to predict in advance. Ultimately what can be built will be built, and I believe regulations and policies can have a minimal impact on the capabilities of AI systems in the long run. But like in the nuclear case, the decisions of first-movers, regulations, and research investments we make today can profoundly impact humanity’s future trajectory. It was not pre-ordained that we would live in a world with more than 3,000 thermonuclear missiles ready to launch at a second's notice. It was also not pre-ordained that 75 years after Hiroshima and Nagasaki, only nine countries would have nuclear weapons, and no such weapon has been used in war since. Technology might be inevitable, but the way we use it isn’t.