The Hanson-Yudkowsky AI-Foom Debate

In late 2008, an extensive and long-awaited debate about the Technological Singularity occurred on Overcoming Bias, mainly between Robin Hanson and Eliezer Yudkowsky. It focused on the likelihood of hard AI takeoff ("FOOM"), the need for a theory of Friendliness, and the future of AGI, whole brain emulation, and recursive self-improvement in general. A stand-up debate on the same themes took place at Jane Street Capital on Wednesday June 30 2011. A full transcript of the debate is available here.

An incomplete summary and analysis of the text debate is available here.

The posts constituting this debate have been collected here to make it easier to follow for future audiences, alongside links to the Jane Street debate.

Prologue

• Fund UberTool? by Robin Hanson
• Engelbart as UberTool? by Robin Hanson
• Friendly Teams by Robin Hanson
• Friendliness Factors by Robin Hanson
• The Weak Inside View by Eliezer Yudkowsky
• Setting The Stage by Robin Hanson
• The First World Takeover by Eliezer Yudkowsky
• Abstraction, Not Analogy by Robin Hanson
• Whence Your Abstractions? by Eliezer Yudkowsky

Main sequence

• AI Go Foom by Robin Hanson
• Optimization and the Singularity by Eliezer Yudkowsky
• Eliezer's Meta-Level Determinism by Robin Hanson
• Observing Optimization by Eliezer Yudkowsky
• Life's Story Continues by Eliezer Yudkowsky
• Emulations Go Foom by Robin Hanson
• Brain Emulation and Hard Takeoff by Carl Shulman
• Billion Dollar Bots by James Miller
• Surprised by Brains by Eliezer Yudkowsky
• “Evicting” brain emulations by Carl Shulman
• Cascades, Cycles, Insight... by Eliezer Yudkowsky
• When Life Is Cheap, Death Is Cheap by Robin Hanson
• ...Recursion, Magic by Eliezer Yudkowsky
• Abstract/Distant Future Bias by Robin Hanson
• Engelbart: Insufficiently Recursive by Eliezer Yudkowsky
• Total Nano Domination by Eliezer Yudkowsky
• Dreams of Autarky by Robin Hanson
• Total Tech Wars by Robin Hanson
• Singletons Rule OK by Eliezer Yudkowsky
• Stuck In Throat by Robin Hanson
• Disappointment in the Future by Eliezer Yudkowsky
• I Heart CYC by Robin Hanson
• Is the City-ularity Near? by Robin Hanson
• Recursive Self-Improvement by Eliezer Yudkowsky
• Whither Manufacturing? by Robin Hanson
• Hard Takeoff by Eliezer Yudkowsky
• Test Near, Apply Far by Robin Hanson
• Permitted Possibilities, & Locality by Eliezer Yudkowsky
• Underconstrained Abstractions by Eliezer Yudkowsky
• Beware Hockey Stick Plans by Robin Hanson
• Evolved Desires by Robin Hanson
• Sustained Strong Recursion by Eliezer Yudkowsky
• Friendly Projects vs. Products by Robin Hanson
• Is That Your True Rejection? by Eliezer Yudkowsky
• Shared AI Wins by Robin Hanson
• Artificial Mysterious Intelligence by Eliezer Yudkowsky
• Wrapping Up by Robin Hanson
• True Sources of Disagreement by Eliezer Yudkowsky
• The Bad Guy Bias by Robin Hanson
• Disjunctions, Antipredictions, Etc. by Eliezer Yudkowsky
• Are AIs Homo Economicus? by Robin Hanson
• Two Visions Of Heritage by Robin Hanson
• The Mechanics of Disagreement by Eliezer Yudkowsky

Conclusion

• What Core Argument? by Robin Hanson
• What I Think, If Not Why by Eliezer Yudkowsky
• Not Taking Over the World by Eliezer Yudkowsky

Postscript

• We Agree: Get Froze by Robin Hanson
• You Only Live Twice by Eliezer Yudkowsky
• Foom Debate, Again by Robin Hanson

Jane Street debate

• Debating Yudkowsky by Robin Hanson
• Video with better quality audio 1.57G
• Poor quality WMA audio 97M
• Poor quality MP3 audio 97M
• Full Transcript
• Partial transcript by Chris Hallquist

eBook

An eBook version of the debate is available in various formats (PDF, EPUB, and MOBI) here. It contains:

• the original series of blog posts,
• a transcript of a 2011 in-person debate between Hanson and Yudkowsky,
• a summary of the debate written by Kaj Sotala, and
• a 2013 technical report on AI takeoff dynamics (“intelligence explosion microeconomics”) written by Yudkowsky.

See also

• Intelligence explosion
• Outside view