My intuition (not rigorous) is there a multiple levels in the consequentialist/deontoligical/consequentialist dealio.

I believe that unconditional friendship is approximately something one can enter into, but one enters into it for contingent reasons (perhaps in a Newcomb-like way – I'll unconditionally be your friend because I'm betting that you'll unconditionally be my friend). Your ability to credibly enter such relationships (at least in my conception of them) is dependent on you not starting to be more "conditional" because you doubt that the other person is also being there. This I think is related to not being a "fair weather" friend. I continue to be your friend even when it's not fun (you're sick, need taking care of whatever) even if I wouldn't have become your friend to do that. And vice versa. Kind of a mutual insurance policy.

Same thing could be with contracts, agreements, and other collaborations. In a Newcomb-like way, I commit to being honest, being cooperative, etc to a very high degree even in the face of doubts about you. (Maybe you stop by the time someone is threatening your family, not sure what Ben, et al, think about that.) But the fact I entered into this commitment was based on the probabilities I assigned to your behavior at the start.

I see interesting points on both sides here. Something about how this comment(s) is expressed makes me feel uncomfortable, like this isn't the right tone for exploring disagreements about correct moral/cooperative behavior, it at least it makes it a lot harder for me to participate. I think it's something like it feels like performing moral outrage/indignation in a way that feels more persuadey than explainy, and more in the direction of social pressure, norms-enforcery. The phrase "shame on you" is a particularly clear thing I'll point at that makes me perceive this.

I was going to write stuff about integrity, and there's stuff to that, but the thing that is striking me most right now is that the whole effort seemed very incompetent and naive. And that's upsetting.

I am now feeling uncertain about the incompetence and naivety of it. Whether this was the best move possible that failed to work out, or best move possible that actually did get a good outcome, or a total blunder is determined by info I don't have.

I have some feeling of they were playing against a higher-level political player which both makes it hard but also means they needed to account for that? Their own level might be 80+th percentile in reference class of executive/board type-people, but still lower than Sam.

The piece that does seem most like they really made a mistake was trying to appoint an interim CEO (Mira) who didn't want the role. It seems like before doing that, you should be confident the person wants it.

I've seen it raised that the board might find the outcome to be positive (board stays independent even if current members leave?). If that's true, does change the evaluation of the competence. Feels hard for me to confidently judge that, though my gut sense is Sam got more of what he wanted/common knowledge of his sway than others.

Styling of the headers in this post is off and makes it harder to read. Maybe the result of a bad copy/paste?

These recent events have me thinking the opposite: policy and cooperation approaches to making AI go well are doomed – while many people are starting to take AI risk seriously, not enough are, and those who are worried will fail to restrain those who aren't (where not being risked in a consequence of humans often being quite insane when incentives are at play). The hope lies in somehow developing enough useful AI theory that leading labs adopt and resultantly build an aligned AI even though they never believed they were going to cause AGI ruin.

And so maybe let's just get everyone to focus on the technical stuff. Actually more doable than wrangling other people to not build unsafe stuff.

If he can lead an exodus from OpenAI to Microsoft, he can lead one from Microsoft to somewhere else.

People associated with EA are likely to decide at some point that the normal rules for the organization do not apply to them, if they expect that they can generate a large enough positive impact in the world by disregarding those rules.

I am myself consequentialist at my core, but invoking consequentialism to justify breaking commitments, non-cooperation, theft, or whatever else is just a stupid bad policy (the notion of people doing this generates some strong emotions for me) that as a policy/algorithm, won't result in accomplishing one's consequentialist goals.

I fear what you say is not wholly inaccurate and is true of at least some in EA, I hope though it's not that true of many.

Where it does get tricky is potentially unilateral pivotal acts about which I think go in this direction but also feel different from what you describe.

You were the first, as you guessed.

If I were to do it again, I might include such an option, though I'm still not terribly sad I didn't.

If we really wanted that info, I could  sample and survey people who received the message, looked at it (we have data to know this) and ask them why they didn't vote. My guess is between 1-10% of people who didn't vote because of frame commented about it,, so that's 40 to 400. 

372 people have responded by now out of 2500, so 15%. Let's guess that 50% of people saw it by now, so ~1250 (though could get better data on it).  A third responded if so, which seems great for a poll. Of those the 800 who saw but didn't, I could see 100-400 doing so because the frame didn't really seem right (lining up with the above estimate). Which seems fine. I bet if I'd spent 10x developing the poll, I wouldn't get that number down much, and knowing it with more precision doesn't really help. It's LW, people are very picky and precise (a virtue...but also makes having some nice things hard).

Sorry, I get the point that the option provided doesn't let you mu/reject the frame. It's not clear to me that this is a core framing of Petrov's actions/virtue was conscientious objection or so on.

Beyond that, the survey wasn't aiming to allow people to symbolically act out their responses or to reject the frame in an unambiguous way. Insisting that you get to register that you saw it but didn't like it feels like insisting that you get to participate, but in your own way, rather than simply not engaging if you don't like it. I also feel like if there was an option to conscientious object and you took that, that'd still be within the frame I created for you to do so? But open to being corrected here.