I believe the confusion comes from assuming the current board follows rules rather than doing whatever is most convenient.

The old board was trying to follow the rules, and the people in question were removed (technically were pressured to remove themselves).

I'd agree the OpenAI product line is net positive (though not super hung up on that). Sam Altman demonstrating what kind of actions you can get away with in front of everyone's eyes seems problematic.

Or simply when scaling becomes too expensive.

There's a lot of problems with linking to manifold and calling it "the expert consensus"!

• It's not the right source. The survey you linked elsewhere would be better.
• Even for the survey, it's unclear whether these are the "right" experts for the question. This at least needs clarification.
• It's not a consensus, this is a median or mean of a pretty wide distribution.

I wouldn't belabor it, but you're putting quite a lot of weight on this one point.

I mean it only suggests that they're highly correlated. I agree that it seems likely they represent the views of the average "AI expert" in this case. (I should take a look to check who was actually sampled)

My main point regarding this is that we probably shouldn't be paying this particular prediction market too much attention in place of e.g. the survey you mention. I probably also wouldn't give the survey too much weight compared to opinions of particularly thoughtful people, but I agree that this needs to be argued.

In general, yes - but see the above (I.e. we don't have a properly functioning prediction market on the issue).

Who do you consider thoughtful on this issue?

It's more like "here are some people who seem to have good opinions", and that would certainly move the needle for me.

I mostly try to look around to who's saying what and why and find that the people I consider most thoughtful tend to be more concerned and take "the weak argument" or variations thereof very seriously (as do I). It seems like the "expert consensus" here (as in the poll) is best seen as some sort of evidence rather than a base rate, and one can argue how much to update on it.

That said, there's a few people who seem less overall concerned about near-term doom and who I take seriously as thinkers on the topic. Carl Shulman being perhaps the most notable.

It eliminates all the aspects of prediction markets that theoretically make them superior to other forms of knowledge aggregation (e.g. surveys). I agree that likely this is just acting as a (weirdly weighted) poll in this case, so the biased resolution likely doesn't matter so much (but that also means the market itself tells you much less than a "true" prediction market would).

Fair! I've mostly been stating where I think your reasoning looks suspicious to me, but that does end up being points that you already said wouldn't convince you. (I'm also not really trying to)

Relatedly, this question seems especially bad for prediction markets (which makes me consider the outcome only in an extremely weak sense). First, it is over an extremely long time span so there's little incentive to correct. Second, and most importantly, it can only ever resolve to one side of the issue, so absent other considerations you should assume that it is heavily skewed to that side.