Top postsTop post
The Fermi paradox is premised on expansionism being intrinsic to civilizational development, most explicitly in Hanson's grabby aliens model. There's a Great Filter provided by expansionism itself that doesn't center on scientific and technological progress that I think is worth exploring. It's about the destabilizing nature of expansion events and...
The recently published METR report on the OpenAI/Hugging Face incident is extremely detailed and shocking. But it’s too long. Intelligent public distillations (like Zvi’s) are also too long, while tolerable summaries lose the meat. This is the report you want. Only the action, in chronological order. Let’s go. Quotes are...
TLDR: The recent agent sandbox incidents suggest that models can know what an action would do while failing to treat those consequences as part of the current decision problem, which I’ll call a failure of consequence salience. One possible cause is a causal-horizon mismatch: agent training often presents trajectories that...
In IABIED, the load-bearing argument and, to me, the main contribution of the book, is about ASI motives. There’s more in there, but the thrust of the book is to argue for the truth of a specific conclusion about motives, namely that an ASI’s motives and goals would be completely...