However, I think most people were in fact very concerned about AI existential risk.

Is your sense here "a large majority" or "a small majority"? Just curious about the rough data here. Like more like 55% or more like 80%?

Sure! I was including "setting up a system that bans other people" in my definition here. I am not that familiar with how DSL works, but given that it bans people, and it was set up by Said, felt confident that thereby somehow Said chose to build a system that does ban people.

Though if Said opposes DSL banning people (and he thinks the moderators are making a mistake when doing so) then I would want to be corrected!

Has some things going for it, but probably too vulgar for most occasions, and also IMO implies too much malintent. I like "diagonalization" because it's kind of a thing you can imagine doing. 

I think my current favorite choice is "Leveling" and "starting a leveling war" from poker. Sentences like "man, I feel like this is too much trying to level them" and "this feels like it's starting a leveling war" are decent-ish pointers.

I don't know about Scott. Him being personally active on the site was long before my tenure as admin, and I am not even fully sure how moderation or deletion at the time worked.

I don't think Said ever banned anyone, though he also wrote only a very small number of top-level posts, so there wasn't much opportunity. My guess is he wouldn't have even if he had been writing a lot of top-level posts.

I don't think the central-case valuable PhDs can be bought or sold so I'm not sure what you mean by market value here. If you can clarify, I'll have a better idea whether it's something I'd bet against you on.

I was thinking of the salary premium that having a PhD provides (i.e. how much more people with PhDs make compared to people without PhDs), which of course is measuring a mixture of real signaling value, and simply just measuring correlations in aptitude, but I feel like it would serve as a good enough proxy here at least directionally.

I would bet a fair amount at even odds that Stanford academics won't decline >1 sigma in collective publication impact score like h-index, Stanford funding won't decrease >1 sigma vs Ivy League + MIT + Chicago, Stanford new-PhD aggregate income won't decline >1 sigma vs overall aggregate PhD income, and overall aggregate US PhD income won't decline >1 sigma. I think 1 sigma is a reasonable threshold for signal vs noise.

What's the sigma here? Like, what population are we measuring the variance over? Top 20 universities? All universities? I certainly agree that Stanford won't lose one sigma of status/credibility/etc. as measured in all universities, that would require dropping Stanford completely from the list of top universities. I think losing 1 sigma of standing among top 20 universities, i.e. Stanford moving from something like "top 3" to "top 8" seems plausible to me, though my guess is a bit too intense. 

To be clear, my offered bet was more about you saying that academia at large is "too big to fail". I do think Stanford will experience costs from this, but at that scale I do think noise will drown out almost any signal. 

TBTF institutions usually don't collapse outside strong outside conquest or civilizational collapse, or Maoist Cultural Revolution levels of violence directed at such change, since they specialize in creating loyalty to the institution. So academia losing value would look more like the Mandarin exam losing value, than like Dell Computer losing value.

Hmm, I don't currently believe this, but it's plausible enough that I would want to engage with it in more detail. Do you have arguments for this? I currently expect more of a gradual devaluing of the importance of academic status in society, together with more competition about the relevant signifiers of status creating more noise, resulting in a relationship to academia somewhat more similar (though definitely not all the way there) as pre-WW2 society had to academia (which to my understanding was a much less central role in government and societal decision-making).

I understand that you don't! But almost everyone else who I do think has those attributes does not have those criteria. Like, Scott Alexander routinely bans people from ACX, even Said bans people from datasecretslox. I am also confident that the only reason why you would not ban people here on LW, is because the moderators are toiling for like 2 hours a day to filter out the people obviously ill-suited for LessWrong.

They, being authors themselves, see the author's pain firsthand, but the commenter's feelings are merely an abstract report at most.

I do think there is a bunch of truth to this, but I am active on many other forums, and have e.g. been issued moderation warnings on the EA Forum, so I do experience moderation in other contexts (and of course get blocked on Twitter from time to time). Also, I... think authors are not that much less likely to ban moderators from their posts than other users. Of the maybe 30 users who have ever been banned from other user posts, one of them is a moderator: 

I am sure that if Eliezer was more active on the site, my guess is people would be a bunch more likely to ban him from their posts than they would other people for the same behavior. In general, in my experience, tallest-poppy dynamics are stronger in the rationality community than leadership-deference dynamics.

While I think building safety-adjacent RL envs is worse than most kinds of technical safety work for people who are very high context in AGI safety, I think it's net positive.

I think it's a pretty high-variance activity! It's not that I can't imagine any kind of RL environment that might make things better, but most of them will just be used to make AIs "more helpful" and serve as generic training data to ascend the capabilities frontier. 

Like, yes, there are some more interesting monitor-shaped RL environments, and I would actually be interested in digging into the details of how good or bad some of them would be, but the thing I am expecting here are more like "oh, we made a Wikipedia navigation environment, which reduces hallucinations in AI, which is totally helpful for safety I promise", when really, I think that is just a straightforward capabilities push.

I would bet that the average research impact of SPAR participants is significantly lower than that of MATS

I mean, sure? I am not saying your selection is worse than useless and it would be better for you to literally accept all of them, that would clearly also be bad for MATS.

I think you are prioritizing a different skillset than most mentors that our mentor selection committee rates highly. Interestingly, most of the technical mentors that you rate highly seem to primarily care about object-level research ability and think that strategy/research taste can be learned on the job!

I mean, there are obvious coordination problems here. In as much as someone is modeling MATS as a hiring pipeline, and not necessarily the one most likely to produce executive-level talent, you will have huge amounts of pressure to produce line-worker talent. This doesn't mean the ecosystem doesn't need executive-level talent (indeed, this post is partially about how we need more), but of course large scaling organizations create more pressure for line-working talent. 

Two other issues with this paragraph: 

• Yes, I don't think strategic judgement generally commutes. Most MATS mentors who I think are doing good research don't necessarily themselves know what's most important for the field.
• I agree with the purported opinion that strategy/research taste can often be learned on the job. But I do feel very doomy about recruiting people who don't seem to care deeply about x-risk. I would be kind of surprised if the mentors I am most excited about don't have the same opinion, but it would be an interesting update if so!

Note that I think the pendulum might start to swing back towards mentors valuing high-level AI safety strategy knowledge as the Iterator archetype is increasingly replaced/supplemented by AI. The Amplifier archetype seems increasingly in-demand as orgs scale, and we might see a surge in Connectors as AI agents improve to the point that their theoretical ideas are more testable. Also note that we might have different opinions on the optimal ratio of "visionaries" vs. "experimenters" in an emerging research field.

I don't particularly think these "archetypes" are real or track much of the important dimensions, so I am not really sure what you are saying here.

Ok, having more time today and thinking more about it, I have updated the description of the proof in the infobox! Curious whether it seems better/more accurate to you now.