Matthew Barnett

Someone who is interested in learning and doing good.

My main blog:


Daily Insights


Three reasons to expect long AI timelines

Wow, that chart definitely surprised me. Yes, this caused me to update.

Three reasons to expect long AI timelines

Nuclear power is not 10x cheaper.  It carries large risks so some regulation cannot be skipped.  I concur that there is some unnecessary regulation, but the evidence such as the linked source just doesn't leave "room" for a 10x gain.  Currently the data suggests it doesn't provide an economic gain over natural gas unless the carbon emissions are priced in, and they are not in most countries.

I recommend reading the Roots of Progress article I linked to in the post. Most of the reason why nuclear power is high cost is because of the burdensome regulations. And of course, regulation is not uniformly bad, but it seems from the chart Devanney Figure 7.11 in the article that we could have relatively safe nuclear energy for a fraction of its current price.

Three reasons to expect long AI timelines

Thanks for the useful comment.

You might say "okay, sure, at some level of scaling GPTs learn enough general reasoning that they can manage a corporation, but there's no reason to believe it's near".

Right. This is essentially the same way we might reply to Claude Shannon if he said that some level of brute-force search would solve the problem of natural language translation.

one of the major points of the bio anchors framework is to give a reasonable answer to the question of "at what level of scaling might this work", so I don't think you can argue that current forecasts are ignoring (2).

Figuring out how to make a model manage a corporation involves a lot more than scaling a model until it has the requisite general intelligence to do it in principle if its motivation were aligned.

I think it will be hard to figure out how to actually make models do stuff we want. Insofar as this is simply a restatement of the alignment problem, I think this assumption will be fairly uncontroversial around here. Yet, it's also a reason to assume that we won't simply obtain transformative models the moment they become theoretically attainable.

It might seem unfair that I'm inputting safety and control as an input in our model for timelines, if we're using the model to reason about the optimal time to intervene. But I think on an individual level it makes sense to just try to forecast what will actually happen.

Three reasons to expect long AI timelines

These arguments prove too much; you could apply them to pretty much any technology (e.g. self-driving cars, 3D printing, reusable rockets, smart phones, VR headsets...).

I suppose my argument has an implicit, "current forecasts are not taking these arguments into account." If people actually were taking my arguments into account, and still concluding that we should have short timelines, then this would make sense. But, I made these arguments because I haven't seen people talk about these considerations much. For example, I deliberately avoided the argument that according to the outside view, timelines might be expected to be long, since that's an argument I've already seen many people make, and therefore we can expect a lot of people to take it into account when they make forecasts.

I agree that the things you say push in the direction of longer timelines, but there are other arguments one could make that push in the direction of shorter timelines

Sure. I think my post is akin to someone arguing for a scientific theory. I'm just contributing some evidence in favor of the theory, not conducting a full analysis for and against it. Others can point to evidence against it, and overall we'll just have to sum over all these considerations to arrive at our answer.

Three reasons to expect long AI timelines

I'm uncertain. I lean towards the order I've written them in as the order of relative importance. However, the regulation thing seems like the biggest uncertainty to me. I don't feel like I'm good at predicting how people and government will react to things; it's possible that technological advancement will occur so rapidly and will be celebrated so widely that people won't want it to stop.

This does not work without a drastic reduction in total government expenditure.

I agree, though I'm not sure whether this will always be true. Look at the last section of my post.

the majority of "wealth" is in the form of individuals' earnings potential

I meant wealth as in physical assets, especially capital.

The last section of my post comes to roughly the same conclusion.

Can you clarify what your point is? ChristianKl said that my proposal "creates a strong incentive for the government to make sure that the companies it owns can outcompete the other companies." I partially agreed but gave one reason to disagree; namely, that the government isn't profit-maximizing. 

You seem to be asserting the opposite of what ChristianKl said, that is, that the government has no incentive to outcompete other companies. Can you explain why?

The government might have an incentive to outcompete other companies, but the strength of this incentive is unclear to me. The US government already competes with the private sector in, e.g. delivering mail, but this hasn't lead to the end of FedEx. Unlike private companies, the government generally isn't profit-maximizing in any normal sense, and so it's not clear why they'd benefit from monopolistic practices.

A new acausal trading platform: RobinShould

For you, our patented superintelligent prediction algorithm anticipated that you would want an account, so we already created one for you. Unfortunately, it also predicted that you would make very bad investments in literal galaxy-brain hot takes. Therefore, we decided to terminate your account.

Load More