Riemann Hypothesis Catastrophe

Ruby (+74/-14)
Archelon fix broken link
Archelon (+21/-1) fix broken link
Archelon (+9/-8) fix typo
alexvermeer (+5/-5) solve -> prove
alexvermeer (+2)
alexvermeer (+594) adding this article so I can link to it from the updated version of Creating Friendly AI

AThe Rieman Hypothesis Catastrophe is a hypothesized  "failure of FriendlinessFriendliness"" scenario in which an AI that was asked to prove the Riemann Hypothesis turns all the matter in the solar system into computronium, exterminating humanity along the way.

This scenario is discussed in more detail in "Creating Friendly AI" (see §5.2.6, p. 91)."

This scenario is dscusseddiscussed in more detail in "Creating Friendly AI."

A "failure of Friendliness" scenario in which an AI asked to solveprove the Riemann Hypothesis turns all the matter in the solar system into computronium, exterminating humanity along the way.

A "failure of Friendliness" scenario in which an AI asked to solve the Riemann Hypothesis turns all the matter in the solar system into computronium, exterminating humanity along the way.

This idea possibly originates---with a slightly different phrasing---from Marvin Minsky.

In variant forms, the Riemann Hypothesis Catastrophe is a subclass of the subgoal stomp error, the Devil's Contract problem (both diabolic and golemic), and the "emergent pressures" scenario a la "The Two Faces of Tomorrow."

This scenario is dscussed in more detail in Creating Friendly AI.

See also