Duncan Sabien (Inactive)

Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies

I do not see you as failing to be a team player re: existential risk from AI.

I do see you as something like ... making a much larger update on the bias toward simple functions than I do. Like, it feels vaguely akin to ... when someone quotes Ursula K. LeGuin's opinion as if that settles some argument with finality?

I think the bias toward simple functions matters, and is real, and is cause for marginal hope and optimism, but "bias toward" feels insufficiently strong for me to be like "ah, okay, then the problem outlined above isn't actually a problem."

I do not, to be clear, believe that my essay contains falsehoods that... (read more)

Replying toEliezer and I wrote a book: If Anyone Builds It, Everyone Dies

Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies

LW is giving me issues and I'm having a hard time getting to and staying on the page to reply; I don't know how good my further engagement will be, as a result.

if we don't know how to choose the right data, the network might not generalize the way we want

I want to be clear that I think the only sane prior is on "we don't know how to choose the right data." Like, I don't think this is reasonably an "if." I think the burden of proof is on "we've created a complete picture and constrained all the necessary axes," à la cybersecurity, and that the present state of affairs with... (read more)

Replying toEliezer and I wrote a book: If Anyone Builds It, Everyone Dies

Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies

A gazelle would not predict that a tiger's fur is actually an insanely visible bright color, and a gazelle would be wrong.

unless it was

gazelliezer

(i'm so sorry)

Replying toEliezer and I wrote a book: If Anyone Builds It, Everyone Dies

Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies

I'm replying in an awkward superposition, here:

MIRI staff member, modestly senior (but not a technical researcher), this conversation flagged to my attention in a work Slack msg
The take I'm about to offer is my own, and iirc has not been seen or commented on by either Nate nor Eliezer and my shoulder-copies of them are lukewarm about it at best
Nevertheless I think it is essentially true and correct, and likely at least mostly representative of "the MIRI position" insofar as any single coherent one exists; I would expect most arguments about what I'm about to say to be more along the lines of "eh, this is misleading in X or Y way,

... (read 1734 more words →)

Replying toWhy does Eliezer make abrasive public comments?

Why does Eliezer make abrasive public comments?

To be clear, it's not because we agree with Buck's model. It's more that Eliezer has persistent health and stamina issues and others (Nate, Malo, etc.) need to step up and receive the torch.

(Also "less" doesn't mean "zero".)

Replying toWhy does Eliezer make abrasive public comments?

Why does Eliezer make abrasive public comments?

I personally would not recommend financial support of MIRI, because I'm worried it will amplify net negative communications from him

Small note: Eliezer is largely backing off from direct comms and most of our comms in the next year will be less Eliezer's-direct-words-being-promoted than in the past (as opposed to more). Obviously still lots of Eliezer thoughts and Eliezer perspectives and goals, but more filtered as opposed to less so. Just FYI.

-4

MIRI Comms is hiring

~~See details and apply~~.

2mo

(Application currently closed; email duncan@intelligence.org with questions or concerns.)

In the wake of the success of Nate and Eliezer’s book, If Anyone Builds It, Everyone Dies, we have an opportunity to push through a lot of doors that have cracked open, and roll a lot of snowballs down a lot of hills. 2026 is going to be a year of ambitious experimentation, trying lots of new ways to deliver MIRI ideas and content to newly receptive audiences.

This means ramping up our capacity, particularly in the arena of communications. Our team did an admirable job in 2025 of handling all of the challenges of launching and promoting a book (including helping... (read 676 more words →)

Replying toTruth or Dare

Truth or Dare

It's going to depend a lot on the social bubble/which group of friends. It's not outrageous for the social circles I run in, which are pretty liberal/West Coast, but it would be outrageous for some bubbles I consider to be otherwise fun and fine and healthy.

Mainly it leans into the archetype of games like Truth or Dare, or Hot Seat, which are sort of canonically teenage party games and thus often trying to loosen those particular strictures.

Replying toMIRI’s 2025 Fundraiser

MIRI’s 2025 Fundraiser

Copying over a comment from the EA forum (and my response) because it speaks to something that was in some earlier drafts, that I expect to come up, and that is worth just going ahead and addressing imo.

IMO it would help to see a concrete list of MIRI's outputs and budget for the last several years. My understanding is that MIRI has intentionally withheld most of its work from the public eye for fear of infohazards, which might be reasonable for soliciting funding from large private donors but seems like a poor strategy for raising substantial public money, both prudentially and epistemically.
If there are particular projects you think are too dangerous to

... (read 499 more words →)

Obligated to Respond

5mo

And, a new take on guess culture vs ask culture

Author's note: These days, my thoughts go onto my substack by default, instead of onto LessWrong. Everything I write becomes free after a week or so, but it’s only paid subscriptions that make it possible for me to write. If you find a coffee’s worth of value in this or any of my other work, please consider signing up to support me; every bill I can pay with writing is a bill I don’t have to pay by doing other stuff instead. I also accept and greatly appreciate one-time donations of any size.

There’s a piece of advice I see thrown around on social... (read 3234 more words →)

151

•••

Make More Grayspaces

7mo

I.

You’ve probably seen that scene where someone reaches out to give a comforting hug to the poor sad abused traumatized... (read 3835 more words →)

311

•••

Truth or Dare

9mo

Author's note: This is my apparently-annual "I'll put a post on LessWrong in honor of LessOnline" post. These days, my writing goes on my Substack. There have in fact been some pretty cool essays since last year's LO post.

Structural note:
Some essays are like a five-minute morning news spot. Other essays are more like a 90-minute lecture.

This is one of the latter. It’s not necessarily complex or difficult; it could be a 90-minute lecture to seventh graders (especially ones with the right cultural background).

But this is, inescapably, a long-form piece, à la In Defense of Punch Bug or The MTG Color Wheel. It takes its time. It doesn’t apologize for its meandering (outside... (read 20665 more words →)

263

•••

DunCon @Lighthaven

Thresholding

Duncan Sabien (Inactive), Screwtape

(This is a linkpost for Duncan Sabien's article "Thresholding" which was published July 6th, 2024. I (Screwtape) am crossposting a linkpost version because I want to nominate it for the Best of LW 2024 review - I'm not the original author.)

If I were in some group or subculture and I wanted to do as much damage as possible, I wouldn’t create some singular, massive disaster.

Instead, I would launch a threshold attack.

I would do something objectionable, but technically defensible, such that I wouldn’t be called out for it (and would win or be exonerated if I were called out for it). Then, after the hubbub had died down, I would do it again.... (read 456 more words →)

Review: Conor Moreton's "Civilization & Cooperation"

Author's note: These days, my thoughts go onto my substack instead of onto LessWrong. Everything I write becomes free after a week or so, but it’s only paid subscriptions that make it possible for me to write. If you find a coffee’s worth of value in this or any of my other work, please consider signing up to support me; every bill I can pay with writing is a bill I don’t have to pay by doing other stuff instead. I also accept and greatly appreciate one-time donations of any size.

Dang it, I knew I should have gone with my first instinct, and photocopied the whole book first. But then again, given... (read 11312 more words →)

Social Dark Matter

You know it must be out there, but you mostly never see it.

Author's Note 1: These days, my thoughts go onto my substack instead of onto LessWrong. Everything I write becomes free after a week or so, but it’s only paid subscriptions that make it possible for me to write. If you find a coffee’s worth of value in this or any of my other work, please consider signing up to support me; every bill I can pay with writing is a bill I don’t have to pay by doing other stuff instead. I also accept and greatly appreciate one-time donations of any size.

Author’s Note 2: This essay is not intended to... (read 9930 more words →)

130

378

•••

Killing Socrates

Exposure to Lizardman is Lethal

Or, On The Willful Destruction Of Gardens Of Collaborative Inquiry

One of the more interesting dynamics of the past eight-or-so years has been watching a bunch of the people who [taught me my values] and [served as my early role models] and [were presented to me as paragons of cultural virtue] going off the deep end.

Those people believed a bunch of stuff, and they injected a bunch of that stuff into me, in the early days of my life when I absorbed it uncritically, and as they've turned out to be wrong and misguided and confused in two or three dozen ways, I've found myself wondering what else they were wrong about.

One of... (read 2218 more words →)

147

212