Mateusz Bagiński

Safety researchers should take a public stance

[Co-written by Mateusz Bagiński and Samuel Buteau (Ishual)] TL;DR Many X-risk-concerned people who join AI capabilities labs with the intent to contribute to existential safety think that the labs are currently engaging in a race that is unacceptably likely to lead to human disempowerment and/or extinction, and would prefer an AGI ban[1] over the current path. This post makes the case that such people should speak out publicly[2] against the current AI R&D regime and in favor of an AGI ban[3]. They should explicitly communicate that a saner world would coordinate not to build existentially dangerous intelligences, at least until we know how to do it in a principled, safe way. They could choose to maintain their political capital by not calling the current AI R&D regime insane, or find a way to lean into this valid persona of “we will either cooperate (if enough others cooperate) or win the competition in style (otherwise)”. X-risk-concerned people who have some influence within AI capabilities labs should additionally truthfully state publicly and advocate internally to ensure that the lab lets its employees speak out publicly, as mentioned above, without any official retaliation. If they are unable to make a lab follow this policy, they should state so publicly. X-risk-concerned people in our communities should enforce the norm of praising the heroism of those who [join AI capabilities labs while speaking out publicly on the current mad race], and being deeply skeptical of the motives of those who [join without publicly speaking out]. Not being public about one's views on this hinders the development of common knowledge, nearly guarantees that the exposure to corrupting influence from working inside the lab (which doesn’t depend on whether they publicly speak out) partially reshapes one into a worse version of oneself, and gives an alibi[4] to people who want to join labs for other reasons that would otherwise be condemned by their community. Quotes > Liron

241Sep 19, 2025

Mateusz Bagiński

Message

I endorse and operate by Crocker's rules.

I have not signed any agreements whose existence I cannot mention.

2969

725

Superintelligence Alignment Seminar (1 month focused upskilling)

[Deadline extended until 22nd of March!] Apply to Seminar to Study, Explain, and Try to Solve Superintelligence Alignment Applications for the AFFINE Superintelligence Alignment Seminar are now open, and we invite you to apply. It will take place in Hostačov, near Prague (Czechia), from 28 April to 28 May.[1] We...

Feb 17109

Reasons to sign a statement to ban superintelligence (+ FAQ for those on the fence)

[Context: This post is aimed at all readers[1] who broadly agree that the current race toward superintelligence is bad, that stopping would be good, and that the technical pathways to a solution are too unpromising and hard to coordinate on to justify going ahead.] TL;DR: We address the objections made...

Oct 13, 202583

Safety researchers should take a public stance

Sep 19, 2025241

Counter-considerations on AI arms races

(Work done at Convergence Analysis. Mateusz wrote the post and is responsible for the outline of the argument, many details of which crystallized in conversations with Justin. Thanks to Olga Babeeva for the feedback on this post.) 1. Introduction: Clarifying the DSA-AI theses Over the last decade, AI research and...

May 15, 202524

Comprehensive up-to-date resources on the Chinese Communist Party's AI strategy, etc?

As the title says. I'm more interested in "up-to-date" than "comprehensive".

Apr 18, 202514

Goodhart Typology via Structure, Function, and Randomness Distributions

(Work done at Convergence Analysis. The ideas are due to Justin. Mateusz wrote the post. Thanks to Olga Babeeva for feedback on this post.) In this post, we introduce the typology of structure, function, and randomness that builds on the framework introduced in the post Goodhart's Law Causal Diagrams. We...

Mar 25, 202535

Bounded AI might be viable

(Work done at Convergence Analysis. Mateusz wrote the post and is responsible for most of the ideas with Justin helping to think it through. Thanks to Olga Babeeva for the feedback on this post.) 1. Motivation Suppose the perspective of pausing or significantly slowing down AI progress or solving the...

Mar 6, 202524

Load More (7/17)

LESSWRONG
LW

LESSWRONG
LW

Mateusz Bagiński

Mateusz Bagiński

Mateusz Bagiński

Safety researchers should take a public stance