I have studied AI safety and existential risks for the last four years and, in this post, I share the resources I liked reading/watching the most. I have read/watched all of them. A few of these resources are in French. Not all the articles are peer-reviewed. I will update this post over time.

Articles I liked the most

• AI safety via debate
• Safe uses of AI Oracles
• The Off-Switch Game
• Formalizing Two Problems of Realistic World-Models
• Goal Misgeneralization in Deep Reinforcement Learning
• Concrete Problems in AI Safety
• The Value Learning Problem
• Penalizing side effects using stepwise relative reachability
• Embedded Agency
• AI Safety Gridworlds
• Conservative Agency
• Parametrically Retargetable Decision-Makers Tend To Seek Power
• Tiling Agents for Self-Modifying AI, and the Löbian Obstacle
• Program Equilibrium in the Prisoner’s Dilemma via Löb's Theorem
• AGI Safety Literature Review
• Formalizing Convergent Instrumental Goals
• Eliciting Latent Knowledge: How to tell if your eyes deceive you
• Cooperative Inverse Reinforcement Learning

YouTube channels and videos

• Rational Animations
• Robert Miles AI Safety
• AI In Context
• SciencePetr
• Monsieur Phi
• Siliconversations
• The video Writing Doom – Award-Winning Short Film on Superintelligence by Foregone Films
• The video L'horreur existentielle de l'usine à trombones by EGO
• The playlist L'intelligence artificielle et le machine learning by Science4All

Podcasts and interviews

• 80,000 Hours Podcast
• AI X-risk Research Podcast (AXRP)
• Doom Debates
• Palisade Research's YouTube channel
• Le Futurologue

Introductions

• Technical AI Safety Course - Blue Dot Impact
• Introduction to AI Safety, Ethics and Society - Center for AI Safety
• Iliad Intensive Course Materials

Organizations

• Center for AI Safety (CAIS)
• Centre pour la Sécurité de l'IA (CeSIA)
• Alignment Research Center (ARC)
• Model Evaluation and Threat Research (METR)
• Center for AI Policy (CAIP)
• Palisade Research
• Convergence Analysis
• SaferAI
• Centre for Long-Term Resilience (CLTR)
• Existential Risk Observatory
• Apollo Research
• ControlAI
• Machine Intelligence Research Institute (MIRI)
• UK AI Security Institute (UK AISI)

Interesting blogposts

• Responsible Scaling Policy - Anthropic
• Activating ASL-3 Protections - Anthropic
• Reward is not the optimization target
• Cryptographic Boxes for Unfriendly AI
• Building Safe A.I. A Tutorial for Encrypted Deep Learning

Declarations, petitions, and quotes from experts

• Center of AI Safety's 2023 statement on AI risk
• AI Red Lines
• ControlAI's Compaign Statement
• The Pro-Human AI Declaration
• ControlAI's list of quotes

Declarations and petitions we can sign

• AI Treaty
• Statement on Superintelligence
• ControlAI's Open Statement
• PauseAI Statement
• PauseAI US Petition
• Pause Giant AI Experiments: An Open Letter
• Open Letter to OpenAI
• Ban Superintelligence: Stop AI-Driven Human Extinction Risk
• Artificial Intelligence: Time is running out! - For responsible AI development!
• AI Summits Need to Take Safety Seriously Again
• Stop the Race to Replace Humans

Surveys about existential risks

• Why do Experts Disagree on Existential Risk and P(doom)? A Survey of AI Experts (Severin Field, 2025, 111 participants).
• Future progress in artificial intelligence: A survey of expert opinion (Müller and Bostrom, 2012-2013, 550 participants).
• 2016 Expert Survey on Progress in AI (AI Impacts, 352 participants).
• 2022 Expert Survey on Progress in AI (AI Impacts, 738 participants).
• 2023 Expert Survey on Progress in AI (AI Impacts, 2778 participants).