In their hopes that it’s not too late for course correction around AI, Nate and Eliezer have written a book making the detailed case for this unfortunate reality. Available in September, you can preorder it now, or read endorsements, quotes, and reviews from scientists, national security officials, and more.
It's from a new podcast on Y Combinator, at 35:27:
...Part of what I've been fighting and maybe what has slowed me down somewhat is that I'm a little ... I don't want to make Terminator real, you know? I've been sort of, at least until recent years dragging my feet on AI and humanoid robotics. Then I've sort of came to realization that it's happening whether I do it or not, so you've got really two choices: you can either be a spectator or a participant. Well, I guess I'd rather be a participant than a spectator. So now it's pedal to the metal on humanoid ro
Before you said
"How do we interpret the inner-workings of neural networks." is not a puzzle unless you get more concrete an application of it. For instance an input/output pair which you find surprising and want an interpretation for, or at least some general reason you want to interpret it.
Which seems to imply you (at least 3 hours ago) believed your theory could handle relatively well-formulated and narrow "input/output pair" problems. Yet now you say
...You just keep on treating it like the narrow domain-specific models count as competition when they
We are having another rationalist Shabbat event at Rainbow Star House this Friday. The plan going forward will be to do one most Fridays. Email or DM me for the address if you haven’t been before.
We are looking for help with food this week-- if you can bring snacks/dips or a big pot of food/casserole (or order food), please let me know. These events will only be sustainable for us if we can keep getting help from the community, please pitch in if you can!
What is this event?
At rationalist Shabbat each week, we light candles, sing Landsailor, eat together, and discuss topics of interest and relevance to the rationalist crowd. If you have suggestions for topics, would like to help contribute food, or otherwise assist with organizing, let us know.
This is a kid-friendly event -- we have young kids, so we have space and toys for them to play and hang out while the adults are chatting.
No, it's supposed to be for June 20th, sorry.
Nate and Eliezer’s forthcoming book has been getting a remarkably strong reception.
I was under the impression that there are many people who find the extinction threat from AI credible, but that far fewer of them would be willing to say so publicly, especially by endorsing a book with an unapologetically blunt title like If Anyone Builds It, Everyone Dies.
That’s certainly true, but I think it might be much less true than I had originally thought.
Here are some endorsements the book has received from scientists and academics over the past few weeks:
...This book offers brilliant insights into the greatest and fastest standoff between technological utopia and dystopia and how we can and should prevent superhuman AI from killing us all. Memorable storytelling about past disaster precedents (e.g. the
Slightly derailing the conversation from the OP: I came across this variant on German Amazon: https://www.amazon.de/Anyone-Builds-Everyone-Dies-Superintelligent/dp/1847928935/
It notably has a different number of pages (32 more) and a different publisher. Is this just a different (earlier?) version of the book, or is this a scam?
Several promising software engineers have asked me: Should I work at a frontier AI lab?
My answer is always “No.”
This post explores the fundamental problem with frontier labs, some of the most common arguments in favor of working at one, and why I don’t buy these arguments.
The primary output of frontier AI labs—such as OpenAI, Anthropic, Meta, and Google DeepMind—is research that accelerates the capabilities of frontier AI models and hastens the arrival of superhuman machines. Each lab’s emphasis on alignment varies, but none are on track to solve the hard problems, or to prevent these machines from growing irretrievably incompatible with human life. In the absence of an ironclad alignment procedure, frontier capabilities research accelerates the extinction of humanity. As a very strong default, I...
> frontier labs are only pretending to try to solve alignment
>>This is probably the main driver of our disagreement.
I agree with your diagnosis! I think Sam Altman is a sociopathic liar, so the fact that he signed the statement on AI risk doesn't convince me that he cares about alignment. I feel reasonably confident about that belief. Zvi's series on Moral Mazes apply here: I don't claim that you literally can't mention existential risk at OpenAI, but if you show signs of being earnestly concerned enough about it to interefere with corporate...
The following post discusses my personal experience of the phenomenology of feminising hormone therapy. It will also touch upon my own experience of gender dysphoria.
I wish to be clear that I do not believe that someone should have to demonstrate that they experience gender dysphoria – however one might even define that – as a prerequisite for taking hormones. At smoothbrains.net, we hold as self-evident the right to put whatever one likes inside one's body; and this of course includes hormones, be they androgens, estrogens, or exotic xenohormones as yet uninvented.
I have gender dysphoria. I find labels overly reifying; I feel reluctant to call myself transgender, per se: when prompted to state my gender identity or preferred pronouns, I fold my hands into the dhyana mudra and...
I can't access the paper by Andersen that you discuss, do you know if schizotypy as Andersen understands it would include the "schizoid" personality type or if he'd consider that distinct? Nancy McWilliams, who wrote an interesting piece about her impressions of schizoid personalities as a psychotherapist, commented on p. 199 of her textbook Psychoanalytic Diagnosis that "Our taxonomic categories remain arbitrary and overlapping, and acting as if there are discrete present-versus-absent differences between labels is not usually wise clinically ... Perhaps ...
Yeah, maybe I should have defined "recursive oversight" as "techniques that attempt to bootstrap from weak oversight to stronger oversight." This would include IDA and task decomposition approaches (e.g. RRM). It wouldn't seem to include debate, and that seems fine from my perspective. (And I indeed find it plausible that debate-shaped approaches could in fact scale arbitrarily, though I don't think that existing debate schemes are likely to work without substantial new ideas.)
[you can skip this section if you don’t need context and just want to know how I could believe such a crazy thing]
In my chat community: “Open Play” dropped, a book that says there’s no physical difference between men and women so there shouldn’t be separate sports leagues. Boston Globe says their argument is compelling. Discourse happens, which is mostly a bunch of people saying “lololololol great trolling, what idiot believes such obvious nonsense?”
I urge my friends to be compassionate to those sharing this. Because “until I was 38 I thought Men's World Cup team vs Women's World Cup team would be a fair match and couldn't figure out why they didn't just play each other to resolve the big pay dispute.” This is the one-line summary...
This was pretty combative. I was thinking of saying "sorry for saying this" but that would have been kinda dishonest as I thought it's better to post this as is then not have something like this comment exist, which were the only realistic options. I will, however, acknowledge that this is a skill issue on my part, and I would prefer to be better at communicating non-violently. I also acknowledge that I'm being somewhat mean here, which isn't virtuous. It would make sense if you thought somewhat less of me for that.