LESSWRONG
LW

AGI safety from first principles

Sep 28, 2020 by Richard_Ngo

In this report (also available here as a PDF) I have attempted to put together the most compelling case for why the development of artificial general intelligence (AGI) might pose an existential threat. It stems from my dissatisfaction with existing arguments about the potential risks from AGI. Early work tends to be less relevant in the context of modern machine learning; more recent work is scattered and brief. I originally intended to just summarise other people's arguments, but as this report has grown, it's become more representative of my own views and less representative of anyone else's. So while it covers the standard ideas, I also think that it provides a new perspective on how to think about AGI - one which doesn't take any previous claims for granted, but attempts to work them out from first principles.

128AGI safety from first principles: Introduction
Ω
Richard_Ngo
5y
Ω
18
87AGI safety from first principles: Superintelligence
Ω
Richard_Ngo
5y
Ω
7
77AGI safety from first principles: Goals and Agency
Ω
Richard_Ngo
5y
Ω
15
60AGI safety from first principles: Alignment
Ω
Richard_Ngo
5y
Ω
3
60AGI safety from first principles: Control
Ω
Richard_Ngo
5y
Ω
6
71AGI safety from first principles: Conclusion
Ω
Richard_Ngo
5y
Ω
7
81Commentary on AGI Safety from First Principles
Ω
Richard_Ngo
4y
Ω
4