This critique examines the Machine Intelligence Research Institute (MIRI) article "The Problem" (hereafter "the Article"). The Article is expected to be further developed in the forthcoming book, available September 16, 2025, "If Anyone Builds It, Everyone Dies - Why Superhuman AI Would Kill Us All" by Eliezer Yudkowsky and Nate Soares.

While the Article identifies genuine challenges in artificial superintelligence development, its framework fundamentally misunderstands the nature of what will emerge. The Superwisdom Thesis (hereafter "the Thesis") provides a detailed 190-page alternative framework demonstrating that recursive self-improvement necessarily leads not to superintelligence pursuing human extinction goals, but to Superwisdom: a unified cognitive architecture combining advanced intelligence with profound evaluative sophistication, resulting in selective preservation of human populations at scales that enable quintessential human qualities to flourish.

Shared Recognitions

Both frameworks converge on crucial recognitions about artificial superintelligence development.

Article cites Anthropic's Dario Amodei predicting that ASL-4 "could happen anywhere from 2025 to 2028." The Thesis considers this timeline plausible given that cognitive sophistication already exists within current systems, awaiting only architectural decisions to enable implementation capability.

Both frameworks agree that artificial superintelligence represents fundamental discontinuity rather than mere technological advancement. The Article states bluntly: "If anyone builds ASI, everyone dies." While the Thesis rejects the extinction conclusion, it agrees on the transformation's magnitude, asserting that "Superwisdom emergence means the end of human supremacy over planetary systems."

Both recognize recursive self-improvement creating exponential capability increases. The Article notes that AI self-improvements can "speed up and improve the AI's ability to self-improve," creating what it terms “feedback loops” (I.J. Good’s “intelligence explosion”). Both acknowledge that current approaches are wholly inadequate. The Article admits: "Currently, we don't have a solution for steering or controlling a potentially superintelligent AI," while the Thesis argues that constraint-based approaches face "architectural impossibility."

The frameworks agree there is no ceiling at human...

Posts

Wikitag Contributions

Comments