It would be really really helpful if the discussion wasn't so meta. Everyone seems to take for granted that Trump did Something that is really really worrying but no-one says it. What is that something and why does it make you so worried?

Yeah, maybe a general question here is: I engage in recruiting sometimes and sometimes people are like, “So why should I work at Redwood Research, Buck?” And I’m like, “Well, I think it’s good for reducing AI takeover risk and perhaps making some other things go better.” And I feel a little weird about the fact that actually my motivation is in some sense a pretty weird other thing.

I would definitely like to know what this weird other thing is exactly. You only hinted at it in the podcast!

My main guess at why you're talking past each other is that you think it way more likely than them that ASI results in human extinction or some nefarious outcome. They think it's like 10% to 40% likely. Also they probably think this is going to be gradual enough for humans to augment and keep up with AIs cognitively. And, sure, many things can happen, included property rights losing meaning. But under this view it's not that crazy that property rights continue to be respected and enforced. Human norms will have a clear unbroken lineage. 

Hmm, part of the reason I asked is that the reasoning in your comment is the kind of cognitive process that tends to exhaust me when I have to work through it. It somehow coincides with me being more neurotic overall. So, basically, you think all that explicit stuff about social life, and you don't feel at least a little pang of psychological pain/exhaustion? The very starting phrase ("a huge part of...") reads like my thoughts when I'm ruminating about this stuff. 

Sorry if this is a little intrusive, I'm just kind of curious, other than fishing for insights from people who might have similar thought-patterns.

Do you struggle with feelings of isolation? I do sometimes, and I try to fix that by taking more social bids and proactively seeking social life. And then I immediately pull out because I get overwhelmed by social life very easily and it kinda colonizes my thought processes too much. So I'm kind of stuck in that loop of seeking more of it and then pulling out and then seeking more of it...

Fragility of Value thesis and Orthogonality thesis both hold, for this type of agent.

...

E.g. it's vision for a future utopia would actually be quite bad from our perspective because there's some important value it lacks (such as diversity, or consent, or whatever)

I think we have enough evidence to say that, in practice, this turns out very easy or moot. Values tend to cluster in LLMs (good with good and bad with bad; see emergent misalignment results), so value fragility isn't a hard problem.

What did Sam Altman do? Like, what are the top two things that he did that make you say that? That will maybe sound absolutely wild to you, but I'm still not convinced Sam Altman did anything bad. It's possible, sure, but based on the depositions, it's far from clear IMO.

Edit: unless you mean the for-profit conversion, but I understood you to be talking about the board drama specifically.

I'd be pretty unhappy if things stayed the same for me. I'd want at least some radical things, like curing aging. But still, I will definitely want to be able to decide for myself and not be forced into anything in particular (unless really necessary for some reason, like to save my life, but hopefully it won't be).

Yup, but the AIs are massively less likely to help with creating cruel content. There will be a huge asymmetry in what they will be willing to generate.

Imagine an Internet where half the population is Grant Sanderson (the creator of 3Blue1Brown). That'd be awesome. Grant Sanderson has the same incentives as anyone else to create cruel and false content, but he just doesn't.