A lot of decisions in the AI safety ecosystem — e.g. which projects get funded, who works where, etc. — are shaped by a distinction between "inside the lab" and "outside the lab." I think this is a useful distinction, but we may be led astray if we rely on...
I think that the external AI safety community should prioritise model specs/constitutions over the next 12 months. It shouldn't be our top priority,[1] but it's pretty important[2] and neglected. In this post, I will argue that it's tractable, even if you aren't a lab employee: 1. It’s a natural language...
Thanks to Vivek Hebbar, Fabien Roger, Lukas Finnveden, and Alex Mallen for discussion, and to Tom Davidson, Joe Kwon, and Ryan Greenblatt whose writing I've drawn on liberally. Errors are my own. Introduction Motivation One obstacle to achieving a near-best future is an AI-enabled coup, and one route to an...
Introduction Some people talk about "hard-to-verify tasks" and "easy-to-verify tasks" like these are both natural kinds. But I think splitting tasks into "easy-to-verify" and "hard-to-verify" is like splitting birds into ravens and non-ravens. * Easy-to-verify tasks are easy for the same reason — there's a known short program that takes...
Edit: This post duplicated a shortform I wrote 6 months ago, which I'd forgotten about when going through my drafts yesterday. Apologies! This post supercedes the shortform, but interested readers should check its replies. Timothy Williamson thinks philosophy is unexceptional. Timothy Williamson[1] thinks that philosophy[2] is far less distinct as...
Sometimes people say things like "If the humans and AIs have linear utility in resources, then their interactions are zero-sum". Here, "linear utility in resources" typically means something like: "Supposing AIs already control all the galaxies, then they'd accept a bet with a 60% chance of gaining one more galaxy...
Consider a future with many diverse AIs that need to coordinate with each other, or at least coexist without conflict. Such AIs would need shared values they can coordinate around. According to Hanson's theory, groups of diverse agents facing coordination pressure will tend to sacralize some shared value — seeing...