In 2018 I recall being at a private talk hosted by ~2 people that OpenPhil worked closely with and/or thought of as senior advisors, on AI. It was a confidential event so I can't say who or any specifics, but they were saying that they wanted to take seriously short AI timelines but demanded confidentiality about this, because they felt that being open about their thoughts on this would influence other actors to believe that AI was important to get involved with, but not that the risk was high. I think this confidentiality about their beliefs was a significant part of the dynamics around OpenPhil's beliefs and signaling at the time.
1Some takes:
- To answer the main question, I agree circa 2020 if you had to ascribe beliefs to Open Phil the org it would be something like "~30 year median timelines" and "~10% utter AI ruin". (And I had some visibility on this beyond public info.) I think this throws away tons of detail about actual Open Phil at the time but imo it's a reasonable move to make.
- I don't think Open Phil had a goal of "promoting" those beliefs, and don't recall them doing things that would be reasonably considered to be primarily about "promoting" those beliefs.
- Obviously they published reports about those topics, but the reason for that is to advance discourse / collective epistemics.
- Iirc, bio anchors was the only one that said ~30 year median timelines, I believe semi-informative priors was significantly longer and compute-centric takeoff was significantly shorter.
- I don't think these beliefs made much of a difference to their grant making on object level technical research (relative to beliefs of shorter timelines / higher probability of ruin), just because people almost always want to act on shorter timelines / higher ruin probabilities (than 10%) because those worlds are easier to influence. This is s
3
2
1Ah, I realized there was something else I should have highlighted. You mention you care about pre-ChatGPT takes towards shorter timelines -- while compute-centric takeoff was published two months after ChatGPT, I expect that the basic argument structure and conclusions were present well before the release of ChatGPT.
While I didn't observe that report in particular, in general Open Phil worldview investigations took > 1 year of serial time and involved a pretty significant and time-consuming "last mile" step where they get a bunch of expert review before publication. (You probably observed this "last mile" step with Joe Carlsmith's report, iirc Nate was one of the expert reviewers for that report.) Also, Tom Davidson's previous publications were in March 2021 and June 2021, so I expect he was working on the topic for some of 2021 and ~all of 2022.
I suppose a sufficiently cynical observer might say "ah, clearly Open Phil was averse to publishing this report that suggests short timelines and intelligence explosions until after the ChatGPT moment". I don't buy it, based on my observations of the worldview investigations team (I realize that it might not have been up to the worldview...
I was heavily involved in the engineering for the compute takeoffs report, so I can confirm the basic model and argument was in place since at least December 2021, in a spreadsheet model Tom put together.
Tom then spent a year implementing new features, collecting and addressing feedback and doing robustness checks before publication.
And the basically final report was ready well before ChatGPT was released.
1(Remember that I only want to defend "worst form of timelines prediction except all the other approaches". I agree this is kind of a crazy argument in some absolute sense.)
So, just so we're on the same page abstractly: Would you agree that updating / investing "a lot" in an argument that's kind of crazy in some absolute sense, would be an epistemic / strategic mistake, even if that argument is the best available specific argument in a relative sense?
Would you agree that updating / investing "a lot" in an argument that's kind of crazy in some absolute sense, would be an epistemic / strategic mistake, even if that argument is the best available specific argument in a relative sense?
Hmm, maybe? What exactly is the alternative?
Some things that I think would usually be epistemic / strategic mistakes in this situation:
- Directly adopting the resulting distribution
- Not looking into other arguments
- Taking actions that would be significantly negative in "nearby" worlds that the argument suggests are unlikely. (The "nearby" qualifier is to avoid problem-of-induction style issues.)
Some things that I don't think would immediately qualify as epistemic / strategic mistakes (of course they could still be mistakes depending on further details):
- Making a large belief update. Generally it seems plausible that you start with some very inchoate opinions (or in Bayesian terms an uninformed prior) and so any argument that seems to have some binding to reality, even a pretty wild and error-prone one, can still cause a big update. (Related: The First Sample Gives the Most Information.)
- This is not quite how I felt about bio anchors in 2020 -- I do think th
Some responses:
RE: AGI ruin probability
I agree with this comment from habryka re: what my view was for most of the relevant time period.
If I were to give a similar range today, my top end would be lower (maybe something in the neighborhood of 50%?), so probably worse according to you than it was during the time period in question.
RE: AGI timelines
I think this post is probably the best public characterization of my state of mind about AI timelines as of November 2021, and broadly for several years before that too. The key part is here:
- I would be at least mildly surprised if transformative AI weren’t developed by 2060. I put the probability of transformative AI by then at 50% (I explain below how the connection works between "mild surprise" and "50%"); I could be sympathetic to someone who said it was 25% or 75%, but would have a hard time seeing where someone was coming from if they went outside that range. More
- I would be significantly surprised if transformative AI weren’t developed by 2100. I put the probability of transformative AI by then at 2 in 3; I could be sympathetic to someone who said it was 1 in 3 or 80-90%, but would have a hard time seeing where someone
It's not obvious to me that Ajeya's timelines aged worse than Eliezer's. In 2020, Ajeya's median estimate for transformative AI was 2050. My guess is that if based on this her estimate for "an AI that can, if it wants, kill all humans and run the economy on its own without major disruptions" would have been like 2056? I might be wrong, people who knew her views better at the time can correct me.
As far as I know, Eliezer never made official timeline predictions, but in 2017 he made an even-odds bet with Bryan Caplan that AI would kill everyone by January 1, 2030. And in December 2022, just after ChatGPT, he tweeted:
Pouring some cold water on the latest wave of AI hype: I could be wrong, but my guess is that we do *not* get AGI just by scaling ChatGPT, and that it takes *surprisingly* long from here. Parents conceiving today may have a fair chance of their child living to see kindergarten.
I think child conceived in December 2022 would go to kindergarten in September 2028 (though I'm not very familiar with the US kindergarten system). Generously interpreting "may have a fair chance" as a median, this is a late 2028 median for AI killing everyone.
Unfortunately, ...
21
1You can do better by saying "I don't know" than by saying a bunch of wrong stuff. My long reply to Cotra was, "You don't know, I don't know, your premises are clearly false, and if you insist on my being Bayesian and providing a direction of predictable error when I claim predictable error then fine your timelines are too long."
31I think an important point is that people can be wrong about timelines in both directions. Anthropic's official public prediction is that they expect "country of geniuses in a data center" by early 2027. I heard that previously Dario predicted AGI to come even earlier, by 2024 (though I can't find any source for this now and would be grateful if someone found a source or corrected me that I'm misremembering). Situational Awareness predicts AGI by 2027. The AI safety community's most successful public output is called AI 2027. These are not fringe figures but some of the most prominent voices in the broader AI safety community. If their timelines turn out to be much too short (as I currently expect), then I think Ajeya's predictions deserve credit for pushing against these voices, and not only blame for stating a too long timeline.
And I feel it's not really true that you were just saying "I don't know" and not implying some predictions yourself. You had the 20230 bet with Bryan. You had the tweet about children not living to see kindergarten. You strongly pushed back against the 2050 timelines, but as far as I know the only time you pushed back agains the very aggressive timelines w...
2I looked at "AI 2027" as a title and shook my head about how that was sacrificing credibility come 2027 on the altar of pretending to be a prophet and picking up some short-term gains at the expense of more cooperative actors. I didn't bother pushing back because I didn't expect that to have any effect. I have been yelling at people to shut up about trading their stupid little timelines as if they were astrological signs for as long as that's been a practice (it has now been replaced by trading made-up numbers for p(doom)).
Huh. I'm fairly confident that we would have chosen a different title if you complained about it to us. We even interviewed you early in the process to get advice on the project, remember? For a while I was arguing for "What Superintelligence Looks Like" as my preferred title, and this would have given me more ammo.
(To be clear, I'm not claiming that we ran the title "AI 2027" by you. I don't think we had chosen a title yet at the time we talked; we just called it "our scenario." My claim is that we were genuinely interested in your feedback & if you had intervened prior to launch to tell us to change the title, we probably would have. We weren't dead-set on the title anyway; it wasn't even my top choice.)
I think your timelines were too aggressive but I wouldn’t worry about the title too much. If by the end of 2027, AI progress is significant enough that no one thinks it’s on track to staying a “normal technology” then I don’t think anyone would hold the 2027 title against you. And if that’s not the case, then titling it AI 2029 wouldn’t have helped.
I think it's good to push back!
IMO clearly if someone believes that timelines are that short, then it makes sense for them to say so loudly and publicly, both so that they can stand corrected when it doesn't happen, and so that people can take the problem with appropriate urgency, so I disagree. And my guess is I have a non-trivial amount of influence on how AI 2027 was done, as well as future projects by the AI Futures Team, and am pretty open to arguments in the space even by your lights, so that such pushback would not have been in vain.
Agree with what Habryka said. Also, Daniel, I, and other AIFP people would update and care about being cooperative / feedback. If anyone is interested in giving feedback on our new scenario about a positive vision post-AGI (about either the content, or the name/branding), please email me.
Also to reiterate: AI 2027 was obviously not a confident prediction of AGI in 2027, it was a scenario where AI happened in 2027, which seems like a plausible and IMO ~modal timeline, and we clearly stated this on the website.
When somebody at least pretending to humility says, "Well, I think this here estimator is the best thing we have for anchoring a median estimate", and I stroll over and proclaim, "Well I think that's invalid", I do think there is a certain justice in them demanding of me, "Well, would you at least like to say then in what direction my expectation seems to you to be predictably mistaken?"
2But I think we shouldn't reward people for only making joking predictions instead of 100-page reports
I don't like this sentiment.
If he predicted ASI in 2028-2030 then I'm not 'punishing' him by believing he did; if he didn't then I'm not 'rewarding' him by believing he didn't.
So your question is whether (with added newline and capitalization for clarity):
any dissenting views from "AI in median >30 years" and "utter AI ruin <10%" (as expressed in the correct directions of shorter timelines and worse ruin chances; and as said before the ChatGPT moment), were permitted to exercise decision-making power over the flow of substantial amounts of funding;
OR if the weight of reputation and publicity of OpenPhil was at any point put behind promoting those dissenting viewpoints
Re the first part:
Open Phil decisions were strongly affected by whether they were good according to worldviews where "utter AI ruin" is >10% or timelines are <30 years. Many staff believed at the time that worlds with shorter timelines and higher misalignment risk were more tractable to intervene on, and so put additional focus on interventions targeting those worlds; many also believed that risk was >10% and that median timeline was <30 years. I'm not really sure how to operationalize this, but my sense is that the majority of their funding related to AI safety was targeted at scenarios with higher misalignment risk and shorter timelines than 10%/30 years.
As an examp...
Section 2.2 in "Some Background..." looks IMO pretty prescient
...The technical advisors I have spoken with the most on this topic are close friends I’ve met through GiveWell and effective altruism: Dario Amodei, Chris Olah and Jacob Steinhardt. They are all relatively junior (as opposed to late-career) researchers; they do not constitute a representative sample of researchers; there are therefore risks in leaning too heavily on their thinking.[...]
There may turn out to be a few broadly applicable AI approaches that lead to rapid progress on an extremely wide variety of intellectual tasks. This intuition seems correlated with (though again, not the same as) an intuition that the human brain makes repeated use of a relatively small set of underlying algorithms, and that by applying the processes, with small modifications, in a variety of contexts, it generates a wide variety of different predictive models, which can end up looking like very different intellectual functions.
[..]Certain areas of AI and machine learning, particularly related to deep neural networks and other deep learning methods, have recently experienced rapid and impressive progress.
[...]Deep learning is a general appro
(As a random reference, I thought Joe's paper about low AI takeover risk was silly at the time, and I think that most people working on grants motivated by AI risk at OP at the time had higher estimates of AI takeover risk. I also thought a lot of takes from the Oxford EAs were pretty silly and I found them frustrating at the time and think they look worse with hindsight. Obviously, many of my beliefs at many of these time periods also look silly in hindsight.)
Okay, so it sounds like you're saying that the claims I asserted aren't cruxy for your claim you wanted contradicted?
I definitely don't think that Open Phil thought of "have more people take MIRI seriously" as a core objective, and I imagine that opinions on whether "people take MIRI more seriously" is good would depend a lot on how you operationalize it.
I think that Open Phil proactively tried to take a bunch of actions based on the hypothesis that powerful AI would be developed within 20 years. I think the situation with the sinking ship is pretty disanalogous—I think you'd need to say that your guy in the expensive suit was also one of the main people who was proactively taking actions based on the hypothesis that the ship would sink faster.
One issue among others is that the kind of work you end up funding when the funding bureaucrats go to the funding-seekers and say, "Well, we mostly think this is many years out and won't kill everyone, but, you know, just in case, we thought we'd fund you to write papers about it" tends to be papers that make net negative contributions.
I think this is a pretty poor model of the attitudes of the relevant staff at the time. I also think your disparaging language here leads to your comments being worse descriptions of what was going on.
1I expect it's a combination of selection effects and researchers knowing implicitly where their bread is buttered; I have no particular estimate of the relative share of these effects, except that they are jointly sufficient that, eg, a granter can hire what advertises itself as a group of superforecasters, and get back 1% probability on AI IMO gold by 2025.
That sounds wild to me, given that the superforecasters believed much less in fast AI progress (and in doom) than OpenPhil staff and the "subject matter experts" who the superforecasters could talk with.
Like, in 2020, bio anchors publicly predicted $1B training runs in 2025. In 2022, the superforecasters predicted that the largest training runs in 2024 would be $35M, in 2030 would be $100M, and in 2050 would be $300M.
(And for the IMO gold number in particular, if I had to guess what OP's view was, I would base that on Paul's 8%. Which is 3/4 of the way from 1% to your own 16%, in log-odds.)
If the superforecasters were biasing their views towards OP, then they should have been way more bullish. If OP's process was selecting for forecasters who agreed more with their own views, they would've selected forecasters who were more bullish.
I think the simpler hypothesis is that the wider world, including superforecasters among them, massively underestimated 2020s AI progress.
(This is consistent with the fact that OP advisors got outsized investment returns by betting on faster AI progress than the markets expected. It's also consistent with Jacob Steinhardt's own attempt at commissioning forecasts, which also produced huge underestimates. I think this wasn't funded by OP, though Jacob was an OP technical advisor at the time.)
1Noted. I think you are overlooking some of the dynamics of the weird dance that a bureaucratic institution does around pretending to be daring while their opinions are in fact insufficiently extreme; eg, why when OpenPhil ran a "change our views" contest, they predictably awarded all of the money to critiques arguing for longer timelines and lower risk, even though reality was in the opposite direction of their opinions from that. Just like OpenPhil predictably gave all the money to "we need two Stalins" critiques of them in the contest, OpenPhil might have managed to communicate to the 'superforecasters' or their institutions that the demanded apparent disagreement with OpenPhil's overt forecast was in the "we need two Stalins" direction of longer timelines and lower risks.
Or to rephrase: If I can look at the organizational dynamics and see it as obvious in advance that OpenPhil's "challenge our worldviews" contest would award all the money to people arguing for longer timelines and lower risk, (despite reality lying in the opposite direction, according to those people's own later updates, even); then maybe the people advertising themselves as producing superforecaster reports,...
I think metaculus and (especially) manifold samples their users disproportionately from AI-risk concerned rationalists and EAs, and relatedly also from people who work in AI. So I'm not that surprised if their aggregated opinions on AI are better than superforecasters. (Although I was pretty surprised by how bad the superforecasters were on some of the questions, in particular the compute spend one.)
Actually, though: what were you referencing with your original claim? (I.e. "get back 1% probability on AI IMO gold by 2025".) I assumed it was from the x-risk persuasion tournament. But page 627-628 says that the superforecasters' 5th percentile for IMO gold was 2025. So they assigned at least 5% that the IMO would get beaten by 2025.
2I'd like to point out that Ajeya Cotra's report was about "transformative AI", which had a specific definition:
...I define “transformative artificial intelligence” (transformative AI or TAI) as “software” (i.e. a computer program or collection of computer programs) that has at least as profound an impact on the world’s trajectory as the Industrial Revolution did. This is adapted from a definition introduced by CEO Holden Karnofsky in a 2016 blog post.
How large is an impact “as profound as the Industrial Revolution”? Roughly speaking, over the course of the Industrial Revolution, the rate of growth in gross world product (GWP) went from about ~0.1% per year before 1700 to ~1% per year after 1850, a tenfold acceleration. By analogy, I think of “transformative AI” as software which causes a tenfold acceleration in the rate of growth of the world economy (assuming that it is used everywhere that it would be economically profitable to use it).
Currently, the world economy is growing at ~2-3% per year, so TAI must bring the growth rate to 20%-30% per year if used everywhere it would be profitable to use. This means that if TAI is developed in year Y, the entire world economy would more
I would like to bring forward from 2017 the paper from Ministry of Foreign Affairs of Finland, done with cooperation of Global Priorities Project and Future of Humanity Institute, University of Oxford
Existential Risk Diplomacy and Governance, 1.1.4 Artificial Intelligence, page 9
...1.1.4 Artificial intelligence Currently, artificial intelligence can outperform humans in a number of narrow domains, such as playing chess and searching data. As artificial intelligence researchers continue to make progress, though, these domains are highly likely to grow in number and breadth over time. Many experts now believe there is a significant chance that a machine superintelligence – a system that can outperform humans at all relevant intelligence tasks – will be developed within the next century. In a 2014 survey of artificial intelligence experts, the median expert estimated that there is a 50% chance of human-level artificial intelligence by 2040, and that once human-level artificial intelligence is achieved, there is a 75% chance of superintelligence in the following 30 years.44 Although small sample size, selection bias, and the unreliability of subjective opinions mean that these estimates
1At a meta level, "publishing, in 2025, a public complaint about OpenPhil's publicly promoted timelines and how those may have influenced their funding choices" does not seem like it serves any defensible goal.
Let's suppose the underlying question is "why did OpenPhil give money to OpenAI in 2017". (Or, conversely, not give money to some other venture in a similar timeframe). Why is this, currently, significantly important? What plausible goal is served by trying to answer this question more precisely?
If it's because they had long timelines, it tells you that short timeline arguments were not effective, which hopefully everyone already knows. This has been robustly demonstrated across most meaningful groups of people controlling either significant money or government clout. It is not information. I would not update on this.
If they did this because they had short timelines, they believed in whatever Sam was selling for that. I would not update on this either. It is hopefully well understood, by now, that Sam is good at selling things. "You could parachute him into an island full of cannibals and come back in 5 years and he'd be the king."
If they did this for non-timeline reasons, I m...
1People ask me questions. I answer them honestly, not least because I don't have the skill to say "I'm not answering that" without it sending some completely different set of messages. Saying a bunch of stuff in private without giving anyone a chance to respond to what I'm guessing about them is deontologically weighed-against by my rules, though not forbidden depending on circumstances. I do not do this in hopes any good thing results, but then acts with good consequences are few and far between in any case, these days.
Clearly there is some value to thinking about past mistakes and getting an accurate retelling of history. In the best case you can identify the mistakes that generated those bad ideas in the past and fix them.
In late 2022, Karnofsky wrote:
I don’t think we’re at the point of having much sense of how the hopes and challenges net out; the best I can do at this point is to say: “I don’t currently have much sympathy for someone who’s highly confident that AI takeover would or would not happen (that is, for anyone who thinks the odds of AI takeover … are under 10% or over 90%).”
I think this is later than what you're asking about; I also would guess that this was Karnofsky's private belief for a while before publishing, but I'm not sure at what time.
Hmm, my sense is Holden was meaning "AI Takeover" roughly in the "AI Ruin" sense, and as such the lower bound here did seem like a helpful thing to say (while I found this upper bound a weirdly unhelpful thing to say).
Like, I think if asked to operationalize Holden at the time would have clarified that by "AI Takeover" he means something that is really bad and catastrophic by his lights.
3Suppose you are correct and that OpenPhil did indeed believe in long timelines pre-ChatGPT. Does this reflect badly on them? It seems like a reasonable prior to me, and many senior researchers even within OA were uncertain that the their methods would scale to more powerful systems.
I think if they sponsored Cotra's work and cited it, this reflects badly on them. More on them than on Cotra, really; I am not a fan of the theory that you blame the people who were selected to have an opinion or incentivised to have an opinion, so much as the people who did the selection and incentivization. See https://www.lesswrong.com/posts/ax695frGJEzGxFBK4/biology-inspired-agi-timelines-the-trick-that-never-works, which I think stands out as clearly correct in retrospect, for why their analysis was obviously wrong at the time. And I did in that case take the trouble to explain why their whole complicated analysis was bogus, and my model is that this clearly-correct-in-retrospect critique had roughly zero impact or effect on OpenPhil; and that is what I expected and predicted in advance, which is why I did not spend more effort trying to redeem an organization I modeled as irredeemably broken.
4Do you find Daniel Kokotajlo’s subsequent work advocating for short timelines valuable? I ask because I believe that he sees/saw his work as directly building on Cotra’s[1].
I think the bar for work being a productive step in the conversation is lower than the bar for it turning out to be correct in hindsight or even its methodology being highly defensible at the time.
Is your position more, “Producing such a model was a fine and good step in the conversation, but OP mistakenly adopted it to guide their actions,” or “Producing such a model was always going to have been a poor move”?
- ^
I remember a talk in 2022 where he presented an argument for 10 year timelines, saying, “I stand on the shoulders of Ajeya Cotra”, but I’m on mobile and can’t hunt down a source. Maybe @Daniel Kokotajlo can confirm or disconfirm.
2At the time, iirc, I went through Ajeya's spreadsheet and thought about each parameter and twiddled them to be more correct-according-to-me, and got something like median 2030 at the end.
If you can get that or 2050 equally well off yelling "Biological Anchoring", why not admit that the intuition comes first and then you hunt around for parameters you like? This doesn't sound like good methodology to me.
2I don't think the intuition came first? I think it was playing around with the model that caused my intuitions to shift, not the other way around. Hard to attribute exactly ofc.
Anyhow, I certainly don't deny that there's a big general tendency for people to fit models to their intuitions. I think you are falsely implying that I do deny that. I don't know if I've loudly stated it publicly before but I definitely am aware of that and have been for years, and I'm embracing it in fact--the model is a helpful tool for articulating and refining and yes sometimes changing my intuitions, but the intuitions still play a central role. I'll try to state that more loudly in future releases.
I think OpenPhil was guided by Cotra's estimate and promoted that estimate. If they'd labeled it: "Epistemic status: Obviously wrong but maybe somebody builds on it someday" then it would have had a different impact and probably not one I found objectionable.
Separately, I can't imagine how you could build something not-BS on that foundation and if people are using it to advocate for short timelines then I probably regard that argument as BS and invalid as well.
1Except that @Daniel Kokotajlo wrote an entire sequence where the only post published after the ChatGPT moment is this one. Kokotajlo's sequence was supposed to explain that Cotra's distribution of training compute for a TAI created by 2020's ideas is biased towards requiring far more compute than is actually needed.
Kokotajlo's quote related to Cotra's errors
Ajeya's timelines report is the best thing that's ever been written about AI timelines imo. Whenever people ask me for my views on timelines, I go through the following mini-flowchart:
1. Have you read Ajeya's report?
--If yes, launch into a conversation about the distribution over 2020's training compute and explain why I think the distribution should be substantially to the left, why I worry it might shift leftward faster than she projects, and why I think we should use it to forecast AI-PONR instead of TAI.
--If no, launch into a conversation about Ajeya's framework and why it's the best and why all discussion of AI timelines should begin there.
However, Kokotajlo's comment outright claims that everyone's timelines should be a variation of Cotra's model.
Kokotajlo praising Cotra's model
So, why do I think it's the best? Well,
31Not an answer to the question, but I think it's worth noting that people asking for your opinion on EA may not be precise with what question they ask. For example, it's plausible to me that someone could ask "has EA been helpful" when their use case for the info is something like "would a donation to EA now be +EV", and not be conscious of the potential difference between the two questions.
I think mainly you're asking about OP in particular, but a side question:
Who is 'Oxford EA'? I definitely interacted with many Oxford-based EA(-adjacent) people, though only since 2022 (pre chatGPT), and the range of views and agendas was broad, and included 'AI soon, very deadly'. I'd guess you mean some more specific (perhaps a funding- or otherwise-rich) smaller group, and I can believe that earlier views were differently distributed.
At many points now, I've been asked in private for a critique of EA / EA's history / EA's impact and I have ad-libbed statements that I feel guilty about because they have not been subjected to EA critique and refutation. I need to write up my take and let you all try to shoot it down.
Before I can or should try to write up that take, I need to fact-check one of my take-central beliefs about how the last couple of decades have gone down. My belief is that the Open Philanthropy Project, EA generally, and Oxford EA particularly, had bad AI timelines and bad ASI ruin conditional probabilities; and that these invalidly arrived-at beliefs were in control of funding, and were explicitly publicly promoted at the expense of saner beliefs.
An exemplar of OpenPhil / Oxford EA reasoning about timelines is that, as late as 2020, their position on timelines seemed to center on Ajeya Cotra's "Biological Timelines" estimate which put median timelines to AGI at 30 years later. Leadership dissent from this viewpoint, as I recall, generally centered on having longer rather than shorter median timelines.
An exemplar of poor positioning on AI ruin is Joe Carlsmith's "Is Power-Seeking AI an Existential Risk?" which enacted a blatant Multiple Stage Fallacy in order to conclude this risk was ~5%.
I recall being told verbally in person by OpenPhil personnel that Cotra and Carlsmith were representative of the OpenPhil view and would be the sort of worldview that controlled MIRI's chances of getting funding from OpenPhil, i.e., we should expect funding decisions to be premised on roughly these views and try to address ourselves to those premises if we wanted funding.
In recent personal conversations in which I exposited my current fault analysis of EA, I've heard people object, "But this wasn't an official OpenPhil view! Why, some people inside OpenPhil discussed different views!" I think they are failing to appreciate the extent to which mere tolerance of dissenting discussion is not central, in an organizational-psychology analysis of what a large faction actually does. But also, EAs have consistently reacted with surprised dismay when I presented my view that these bad beliefs were in effective control. They may have better information than I did; I was an outsider and did not much engage with what I estimated to then be a lost cause. I want to know the true facts of OpenPhil's organizational history whatever they may be.
I therefore throw open to EAs / OpenPhil personnel / the Oxford EAs, the question of whether they have strong or weak evidence that any dissenting views from "AI in median >30 years" and "utter AI ruin <10%" (as expressed in the correct directions of shorter timelines and worse ruin chances; and as said before the ChatGPT moment), were permitted to exercise decision-making power over the flow of substantial amounts of funding; or if the weight of reputation and publicity of OpenPhil was at any point put behind promoting those dissenting viewpoints (in the correct direction, before the ChatGPT moment).
This to me is the crux in whether the takes I have been giving in private were fair to OpenPhil. Tolerance of verbal discussion of dissenting views inside OpenPhil is not a crux. EA forum posts are not a crux even if the bylines include mid-level OpenPhil employees.
Public statements saying "But I do concede 10% AGI probability by 2036", or "conditional on ASI at all, I do assign substantial probability to this broader class of outcomes that includes having a lot of human uploads around and biological humans thereby being sidelined", is not something I see as exculpatory; it is rather a clear instance of what I see as a larger problem for EA and a primary way it did damage.
(Eg, imagine that your steamship is sinking after hitting an iceberg, and you are yelling for all passengers to get to the lifeboats. As it seems like a few passengers might be starting to pay some little attention, somebody wearing a much more expensive and serious-looking suit than you can afford, stands up and begins declaiming about how their own expert analysis does suggest a 10% chance that the ship takes on enough water to sink as early as the next week; and that they think this has a 25% chance of producing a broad class of genuinely attention-worthy harms, like many passengers needing to swim to the ship's next destination.)
I have already asked the shoggoths to search for me, and it would probably represent a duplication of effort on your part if you all went off and asked LLMs to search for you independently. I want to know if insiders have contrary evidence that I as an outsider did not know about. If my current take is wrong and unfair, I want to know it; that is not the same as promising to be easy to convince, but I do want to know.
I repeat: You should understand my take to be that of an organizational-psychology cynic who is not per se impressed by the apparent tolerance of dissenting views, people invited to give dissenting talks, dissenters still being invited to parties, et cetera. None of that will surprise me. I do not view it as sufficient to organizational best practices. I will only be surprised by the demonstrated past pragmatic power to control the disposition of funding and public promotion of ideas, contrary to "AGI median in 30 years or longer" and "utter ruin at 10% or lower", before the ChatGPT moment.
(If you doubt my ability to ever concede to evidence about this sort of topic, observe this past case on Twitter where I immediately and without argument concede that OpenPhil was right and I was wrong, the moment that the evidence appeared to be decisive. (The choice of example may seem snarky but is not actually snark; it is not easy for me to find other cases where, according to my own view, clear concrete evidence came out that I was definitely wrong and OpenPhil definitely right; and I did in that case immediately concede.))