I think that both of these posts seem very confused about the dynamics of who says or thinks what, and I'm pretty sad about these posts.
In general, I'll note that I don't think Rob really knows many of the OP people; I suspect he has spent <40 hours talking to them about any of this possibly ever. (This is in contrast to e.g. Habryka.) I don't know where he's getting his ideas about what the OP people think, but he seems incredibly confused and ignorant. (Eliezer seems similarly ignorant about who believes what.)
'be extremely strategic and game-playing about how you spin the things you say, rather than just straightforwardly reporting on your impressions of things' plus 'opportunistically use Modest Epistemology to dismiss unpalatable views and strategies, and to try to win PR battles'.
I don't really think this is true
Dario and a cluster of Open-Phil-ish people seem to have a very strange and perverse set of views
I wish Rob would be clear who he was referring to. Dario has beliefs that seem to me very different from most people who worked on the 2022 AI misalignment risk efforts at Open Phil. (I'm thinking of people like Holden Karnofsky, Ajeya Cotra, Joe Carlsmith, Lukas Finnveden, Tom Davidson. I'll refer to this as "OP AI people" despite the fact that none of them work at Coefficient Giving (which OP renamed to).) Maybe Rob is talking about what Alexander Berger thinks?
(at least insofar as their public statements to date represent their actual view of the situation):
I think both Dario and Open Phil staff have been reasonably honest about their beliefs about catastrophic misalignment risk publicly, I think that Dario genuinely thinks it's <5% and the OP AI people generally think it's higher. (Tbc I think Dario's take here is very bad!)
1. AI is going to become vastly superhuman in the near future; but being a good scientist means refusing to speculate about the potential novel risks this may pose. Instead, we should only expect risks that we can clearly see today, and that seem difficult to address today.
This is a reasonable statement of (a simple version of) the Dario/Jared/Anthropic position, but not the OP AI person position. The OP AI people were worried about AI misalignment and ASI enough to try to think it through in detail starting many years ago!
If there is some argument for why a problem P might only show up at a higher capability level, or some argument for why a solution S that works well today will likely stop working in the future... well, those are just arguments. Arguments have a terrible track record in AI; the field is full of surprises. So we should stick to only worrying about things when the data mandates it. This is especially important to do insofar as it will help us look more credible and thereby increase our political power and influence.
This is not what the OP people think, e.g. see 1 2 3. It's a reasonable description of what Dario/Jared say.
2. When it comes to technical solutions to AI, the burden of proof is on the skeptic: in the absence of proof that alignment is intractable, we should behave as though we've got everything under control. At the same time, when it comes to international coordination on AI, we will treat the burden of proof as being on the non-skeptic. Absent proof that governments can coordinate on AI, we should assume that they can't coordinate. And since they can't coordinate, there's no harm in us doing a lot of things to make coordination even harder, to make our lives a bit more convenient as we work on the technical problems.
This is not what the OP people think. I think it's somewhat reasonable to accuse Anthropic of this.
3. In general, people worried about AI risk should coordinate as much as possible to play down our concerns, so as not to look like alarmists. This is very important in order to build allies and accumulate political influence, so that we're well-positioned to act if and when an important opportunity arises.
I've never felt any pressure to play down my concerns from the OP people. For example, I've been in a lot of discussions about whether it's better for MIRI to be more or less powerful or influential. To me, the main argument that it's bad for MIRI to be more influential isn't that MIRI is making a mistake by openly saying that risk is high. It's that MIRI has beliefs about x-risk that are wrong on the merits which lead them to making unpersuasive arguments and bad recommendations, and they're in some ways incompetent at communicating.
And I think this is not very representative of what Ant thinks. E.g. they don't really think of themselves as coordinating with other AI-safety-concerned people.
If you're claiming that now is an important opportunity, and that we should be speaking out loudly about this issue today... well, that sounds risky and downright immodest. Many things are possible, and the future is hard to predict! Taking political risks means sacrificing enormous option value. The humble and safe thing to do is to generally not make too much of a fuss, and just make sure we're powerful later in case the need arises.
This is somewhere between "strawman" and "just totally confused as a description of what people believe"
Basically everything else in Rob's post seems like a strawman.
Overall, I think this post is extremely confused, and Rob should be ashamed of writing such incredibly strawmanned things about what someone else thinks.
I recommend that people place very little trust in claims Rob makes about what other people believe. As someone who knows and talks regularly to the "Open Phil AI people", I seriously think that Rob has no idea what he's talking about when he ascribes arguments to them.
I guess there's the question of what we are supposed to do if, in fact, the OP people agree with Rob's version of their position but publicly deny that—at that point we'd have to do some brutal adjudication based on confusing private evidence or inferences from public actions and statements. I really don't think that looking into that evidence would support Rob's claims.
I disagree with all of this on the epistemic level of "it's not true", and additionally disagree with your comms strategy of undermining EAs.
I don't really think of Rob or MIRI as having a comms strategy of undermining EAs. I think Rob and Eliezer just say a bunch of false, wrong things about EAs because they're mad at them for reasons downstream of the EAs not agreeing with Eliezer as much as Eliezer and Rob think would be reasonable, and a few other things.
On the epistemic level - I haven't seen EAs (other than SBF) do a lot of lying, equivocating, or even being particularly shy about their beliefs.
Some EAs engage in equivocation and shyness about their beliefs; OP AI people less than many others.
Absent that, I think this "debate" isn't about OpenPhil or Anthropic failing to say they're extremely worried, failing to say that catastrophe is a very plausible outcome, or failing to say that they think slowing down AI would be good if possible.
I think Dario (like various other Anthropic people) does not believe that AI takeover is a very plausible outcome, and I think his position is indefensible on the merits, as are some of his other AI positions (e.g. his skepticism that there are substantial returns to intelligence above the human level, his skepticism that ASI could lead to 2x manufacturing capacity per year). He moderately disagrees with the OP people about this.
And I think any attempt to attack them for this should start with an acknowledgement that MIRI is directly responsible for all of our current problems by doing things like introducing DeepMind to its funders, getting Sam Altman and Elon Musk into AI, and building up excitement around "superintelligence" in Silicon Valley. I think if 2010-MIRI had slightly more strategicness and willingness to ask itself "hey, is this PR strategy likely to backfire?", you might not have told a bunch of the worst people in the world that AI was going to be super-powerful and that whoever invested in it would be ahead in a race that might make them hundreds of billions of dollars (and yes, you did add "and then destroy the world" - but if you had been more strategic, you might have considered that investors wouldn't hear that last part as loudly).
I don't totally understand what point Scott is trying to make here, but I think this point is quite unfair.
(you could argue that you're not against strategicness in general, just talking about this one issue of saying cleanly that AI is very dangerous. But my impression is that Holden, Dario, have said this, many times - see examples above. What they haven't said is "the situation is totally hopeless and every strategy except pausing has literally no chance of working", but that isn't a comms problem, that's because they genuinely believe something different from you.
Agreed
And also, I frequently encountering people who say things like "Scott, I'm glad you wrote about X in way Y - it made me take AI risk seriously, after I'd previously been turned off of it by encountering MIRI". I think a substantial reason that Dario's writing sometimes seems schizophrenic when talking about AI risks is that he's trying to convey that they're serious while also trying to signal "I swear I'm not one of those MIRI people" so that his writing can reach some of the people you've driven away. I don't think you drive them away because you're "honest", I think it's just about normal issues around framing and theory-of-mind for your audience.)
I think Scott is blaming MIRI much too much here. Dario's main difficulty when arguing that he thinks AI will pose huge catastrophic risk in the next few years is that lots of people think this seems implausible on priors, not because those people were specifically turned off by MIRI making related arguments earlier. His core audience has never heard of MIRI.
But my impression is that the rest of the field is executing this portfolio plan admirably, but MIRI and a few other PauseAI people are trying to sabotage every other strategy in the portfolio in the hope of forcing people into theirs.
I think this is an incorrect read. Some people from PauseAI and MIRI criticize AI safety efforts a lot, often in ways I think are really dumb and counterproductive. But I don't think they're doing this as part of a strategy to force people into their strategies; it's because of some combination of them genuinely (but perhaps foolishly) thinking that the other strategies are bad and/or the people executing them are corrupt.
(I think if you guys had your way, Anthropic would never have been founded, no safety-minded people would ever have joined labs, and the current world would be a race between XAI, Meta, and OpenAI, all of which would have a Yann LeCun style approach to safety, and none of which would have alignment teams beyond the don't-say-bad-words level. We wouldn't have the head of the leading AI lab writing letters to policymakers begging them to "jolt awake", we wouldn't have a substantial fraction of world compute going to Jan Leike's alignment efforts, we wouldn't have Ilya sitting on $50 billion for some super-secret alignment project -- just Mark Zuckerberg stomping on a human face forever. In exchange, we would have won a couple more years of timeline, which would have been pointless, because timeline isn't measured in distance from the year 1 AD, it's measured in distance between some level of woken-up-ness and some point of danger, and the woken-up-ness would be pushed forward at the same rate the danger was.)
I disagree in a lot of the claims here about how various aspects of the current situation are good. (E.g. why does he think that Ilya is doing an alignment effort?)
I support your fight-for-a-pause strategy in theory, and I would like to support it with praxis, but right now I feel very conflicted about this, because I worry that any support or oxygen you guys get will be spent knifing other safety advocates, while Sam Altman happily builds AGI regardless.
It's unclear what "you guys" means. I think Pause AI is making a variety of bad strategic choices. I think that knifing other safety advocates is one bad strategic choice, but it's more like a bad choice that is downstream of my main problems with them, rather than my core concern about them. I think Rob is totally unreasonable and I wish he would stop working on AI safety, but I think he's much worse than e.g. MIRI is overall. I think MIRI spends very little of their support on knifing AI safety advocates, they spend almost all of it on advocating for people being scared about misalignment risk and advocating for AI pauses (which I am generally in favor of). Eliezer totally does have a hobby of saying ridiculously strawmanny stuff about OP AI people, which I find pretty extremely annoying, but I don't think it's a big part of his effect on the world.
----
Overall, both posts seem to have substantially inaccurate pictures of what's going on and what various actors think.