LESSWRONG
LW

1045
Wikitags

Black Marble

Edited by gilch, et al. last updated 4th Feb 2024

A Black Marble is a technology that by default destroys the civilization that invents it. It's one type of Existential Risk. AGI may be such an invention, but isn't the only one.

The name comes from a thought experiment by Nick Bostrom, where he described inventions as pulling marbles out of an urn. Most are white (beneficial), some are dangerous or a mixed blessing (usually described as gray or red balls), and some are black (fatal).

Also sometimes phrased as "black ball" when describing the experiment, including by Bostrom himself, but that already means something else when used by itself.

As a hypothetical example, Bostrom asked what would happen to civilization if a weapon of mass destruction comparable to an atomic bomb was much easier to make, on the level of cooking sand in a microwave or something like that. The natural answer being, once knowledge of the technique spreads, any random psychopath can do it, and some do so, and soon thereafter bomb us back into the stone age. Civilization then can't rebuild past the point of microwaves without getting destroyed again, as long as the knowledge persists (making black marbles an extreme type of information hazard). But this scenario is path dependent. One could imagine a different civilization with different capabilities that could survive such knowledge. Perhaps one with a world government (no wars) and a screening for psychopathy, etc. Perhaps a dystopian world panopticon could prevent use. Or, for a space-faring civilization that mostly lives in small independent orbital colonies, everybody already has (and civilization is somehow currently surviving) similarly destructive kinetic attack capabilities so maybe the sand-nukes don't change much.

Subscribe
Discussion
1
Subscribe
Discussion
1
Posts tagged Black Marble
50The Vulnerable World Hypothesis (by Bostrom)
Ben Pace
7y
17
21Absent coordination, future technology will cause human extinction
Jeffrey Ladish
6y
12
44The Dumbest Possible Gets There First
Ω
Artaxerxes
3y
Ω
7
34My thoughts on nanotechnology strategy research as an EA cause area
Ben_Snodin
3y
0
33A Pin and a Balloon: Anthropic Fragility Increases Chances of Runaway Global Warming
avturchin
3y
23
21misc raw responses to a tract of Critical Rationalism
mako yass
5y
52
15Enlightenment Values in a Vulnerable World
Maxwell Tabarrok
3y
6
12The Transparent Society: A radical transformation that we should probably undergo
mako yass
6y
27
12Take 6: CAIS is actually Orwellian.
Ω
Charlie Steiner
3y
Ω
8
7Nuclear Strategy in a Semi-Vulnerable World
Jackson Wagner
4y
1
Add Posts