This is a special post for quick takes by Hayekstan. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Massively undermade point in favour of the belief that AI will be a very big deal: The AI spend is the largest bet in the history of capitalism. A (peacetime) industrial effort is underway which has not had a comparison since the nineteenth century. Many of us are vaguely aware that large sums are being spent, but the scale of the undertaking is astonishing:
- The six largest spenders this year are spending more on Capital expenditures this year than the Russian government, governing 11 timezones and 143 million people, fighting a war on which the fate of its regime probably depends, spent last year.
- Hyperscaler spending (Alphabet, Amazon, Microsoft, Meta and Oracle) will be $785-820bn this year, about 2.4% of US GDP. There are only 11 governments on earth who spend more than that. Next year they are projected to spend about $1.3 trillion dollars.
- Google and Amazon are each spending about 0.6% of US GDP. The Manhattan project and Apollo programme each accounted for about 0.4% of US GDP in their peak years of spending. Each of these companies has announced their intention to spend much more next year.
- This money paying for thousands of data centres, the largest of which consume as much electricity as major cities and cover thousands of acres. Gas pipelines are being re-routed and nuclear power plants re-opened to fuel them. They already consume as much electricity as Britain. The increase in power demand for data centres from 2023 to 2030 will be the equivalent of adding another top ten power consuming country.
This spending signals a few important things about the beliefs of the key decision makers (who are amongst the most successful capital allocators in the world)
- To recoup their spend, AI will soon have to generate far more revenue than it currently does, so they believe very powerful, very useful AI will be achieved.
- Most of the money is being spent on chips which have life spans of about 6 years, so they believe this will be achieved soon.
- These beliefs are held strongly, not weakly. The companies are not placing small bets. They have burned through close to all of their free cash flow, have issued record setting corporate bonds in the US, UK, EU, Switzerland, Australia, Canada. They have spending commitments for beyond this year in the trillions of dollars. etc.
Leaving aside the question of whether ASI can be built safely using anything like the current methods, because of the size of the spending, key decision makers need to achieve very powerful/ profitable AI soon, to justify their spend.This will encourage recklessness, to say the least.
I personally find the trends based argument the strongest argument for concern about AI: Less than a decade ago LLMs couldn’t reliably write a complex sentence, today they are solving millennium problems. Many of us here on LW are persuaded by a priori arguments about the nature of intelligence; that human intelligence is not at the ceiling dictated by our physics, that intelligence and morality do not necessarily go together, that goal directed AIs will pursue instrumental goals at odds with our interests etc. but these arguments clearly leave most of the public cold. I believe there is a subset of people who might be convinced by the spending/ resource argument, which is why I have written takingstock.ai which adds a lot more detail to what I have said above. Presumably very few people will actually read it, but hopefully some parts of it will make its way into the general discourse and add some weight to our concerns.
The counterargument to this is a claim that it's a gamble of desperation. A large part of the reason many of us went into Computer Science in the first place is that every other place an intelligent, agentic young person would typically go had been closed off.
- Politics was seeing the beginning of a gerontocracy that's become even worse in recent years, with outsiders that don't have backing of some kind being gatekept fairly effectively.
- Physical industry was regulated to the point of being nearly impenetrable, and domain experts were not taking on apprentices anyways.
- The military used to accumulate elites and pass them onto high-status jobs after verifying their competence, producing astronauts and presidents, but it's much less known for that now.
A lot of the ascendance of tech in the 2010's was the product of smart young American guys finding the one place that was willing to take them. Naturally, as a consequence of this, it became the locus of a lot of investment, because investors want to put money where all the agentic young men are headed.
Accordingly, society is somewhat hollowed-out. China is surging past the U.S. in terms of physical industry, the military is demoralized[1], and there's a near-universal sense of social decay. If you're in a position of power in America, and you want to stay on top, betting everything on a technology that might be a game-changer is perfectly reasonable, even if you aren't sure that it will pay off.
Concentrated, unprecedented investment can be explained as extreme confidence, but it can also be explained as a Hail Mary, and this is the counter-narrative I've observed.
- ^
Anecdotally, a lot of military men have told me that they don't really see anything worth dying for anymore.
A large part of the reason many of us went into Computer Science in the first place is that every other place an intelligent, agentic young person would typically go had been closed off.
This feels a little incomplete. Why not, say, high finance or consulting?
Compare high finance or consulting to computer science, politics, physical industry, or military. Those two are much more bereft of meaning or reason-to-drive. That's to some degree social stereotype, I don't personally know anyone who had any form of dream of being in finance or consulting. Beyond that, I'd add to the above that computer science is much easier to get into with learning on your own or a degree. So for "decently intelligent, agentic young person" but who isn't passing higher finance puzzles, they can get a good job in programming where otherwise eh.
That is, I'd add "meaning" (or something closer to that) to why. An agentic individual is still driven by what is interesting or useful.
>I don't personally know anyone who had any form of dream of being in finance or consulting.
Anecdataly as someone who went to business school in undergrad these people absolutely do exist. They dream of it for the money, but they still dream of it.
Rightly or wrongly, most young guys see finance as an established class that you can't really break into. You have to be smart, yes, but also charismatic and well-connected. It's for people who have already "made it", rather than smart middle class people who want upward mobility. This is certainly not completely true, but it's true enough to be a significant deterrent.
My list is of institutions that, at some point, offered the same deal as what the military was in the 1970's. Smart kid from flyover tries out, tests well, gets tracked into a productive and lucrative career. The bulk of your available talent comes from pipelines like this[1], so they're a strong predictor of which institutions will still work well a few decades on.
- ^
It's true that upper-class families produce talent too, and the child of two Harvard finance professors is more likely to be brilliant than the child of two electricians, but regression to the mean cuts hard in societies with low social mobility.
Multiple disagreements here:
- This is heavily private, not public investment, evidence against this is about power politics vs. naked ROI
- My general read is that software continues to select mostly for "nerds". It certainly displaces other nerdy endeavors (science, especially in academia, less patient-facing medicine, corporate law, etc.), but politics/physical industry/military generally select for different individuals
- "Nothing worth dying for" is like low fertility - a symptom of high opportunity costs created by social success.
evidence against this is about power politics vs. naked ROI
Power does not have to be a factor - if you anticipate that society is broadly in decline, putting your money into the one investment that might go up in the long run can be a smart move even if the raw probability is low.
Anecdotally, a lot of military men have told me that they don't really see anything worth dying for anymore.
Is this just about the US military? If not, what other countries do you have a sample from?
Yes, it's just about the American military. There was a similar post-WWI decay in the British military, where it stopped being an elite rite of passage, but my post is entirely focused on the American economy.
Nearly everyone agrees at this point AI will be a big deal (at least on level of the internet), but I don't find the capital spend argument sufficient to imply near-term transformative AGI. (In fact, bond yields remain far too low if that were a highly likely future).
- The capital spend is on the order (as % of GDP) of what the US spent to develop railroads.
- Generating far more revenue does not require massively higher capabilities, just diffusion. Just $10/day LLM spend (most SF companies are an order of magnitude higher than that today) for every white collar worker in the world gets to $4 trillion in annual spend
(most SF companies are an order of magnitude higher than that today) for every white collar worker in the world gets to $4 trillion in annual spend
I think you're making a pretty big logical leap between "white collar worker" and "SF companies". A 10x software engineer at Apple might coordinate a group of LLMs and spend $100 a day in tokens, but for an ordinary white collar worker, even the limits of a $20 a month subscription are usually overkill. There's a very narrow subset of jobs that benefit from that kind of very heavy use of frontier-level LLMs.
I strongly disagree here. First off, 10x engineers are pushing $5,000+/day, not "merely" $100/day (my number is for the entire company) and secondly once you move beyond solo projects (broadly defined), there is plenty of fruit to pick that has strong ROI at employee value of time.
As a very simple example, fully automated systems can detect a server error, analyze traces, and issue a PR to fix it, with at most humans reviewing and merging -- and in fact this has been possible reliably since ~Opus 4.6. That burns plenty of tokens, with high productivity uplift, but I suspect few companies have done this.
The top engineers are spending more because they have learned how to use AI effectively faster than the general population. That's just diffusion.
“Nearly everyone agrees that this point AI will be a big deal”
This seems very wrong to me. I think very few people are taking AI even close to appropriately seriously. Many, especially on the left, believe it is all hype, a la crypto. Policy makers are way off the pace. As the essay argues, https://www.takingstock.ai/#what-comes-next if politicians were taking this seriously they would at least be heavily regulating, it if not seeking to nationalise it (if only to consolidate their own power). More broadly I think that even amongst those who are taking it somewhat seriously, most think of it as a “normal technology” and less than a majority would even say it will be internet level.
(In fact, bond yields remain far too low if that were a highly likely future).
It seems to me that the effect of transformative AI on nominal bond yields is pretty ambiguous. You have a bunch of factors pointing in conflicting directions, depending on exactly which form of that hypothesis comes to pass.
There are all the various scenarios where money doesn't matter (whether because outcomes are really bad or really good). There's no obvious bond trading move today if you think those happen.
Assuming a more moderate but transformative impact, I'd imagine that transformative AI is basically deflationary. If AI can replace labor for a small fraction of the price, then prices of outputs should generally fall. And if you expect substantial deflation, then current nominal bond yields could actually represent sky-high real yields.
I think the point that the AI spend is an enormous bet has been made very very many times, but almost always when I see it it's from people saying "look at the crazy things the AI companies and their investors are doing -- they would only make sense if AI is going to completely transform the world, which of course it obviously isn't, ha ha ha".
Is this an undermade point? Outside of LessWrong and related spaces, there is absolutely nonstop talk about the level of investment in AI and what it will take to pay it off. Estimating off the cuff, I'd say the lead story on the Financial Times website is AI-investment related about half the time over the last couple months.
I’m not convinced that the lifetime of the chips is reasonable upper bound on the time to payoff.
I’m not sure this implies confidence in the bet versus a willingness to take risky bets. Regardless, it is indeed staggering. The upside of which (imo) is that it does seem like a precarious situation, i.e. if one or so training runs fails it might bankrupt the industry. The downside of which is that if governments decide to heavily regulate this technology it seems pretty likely to cause a recession, which seems like a reason governments might be hesitant to.
- To recoup their spend, AI will soon have to generate far more revenue than it currently does, so they believe very powerful, very useful AI will be achieved.
I would push back a little bit on this particular claim. It's not so much that they think it will be a very big deal in the ASI sense. I don't think they actually believe strongly in ASI at all. I do think that they think it will be a big deal existentially for their companies. E.g., Google understands that if Anthropic or OpenAI make an agent that is is a better search experience than Google search, that's the end of Google. For example: I have already stopped using google search myself and rely entirely on AI agents searching the web for me.
It's because of these zero-sum competitive dynamics that we have this spending moreso than the eventual promises of superintelligence itself in my view. Returns on investment are nice, but what's more important is survival. And microsoft, meta, google have no choice but to compete on AI as if it will be the death of them if they don't. Because it will be the death of them if they don't compete.
These companies can spend as much on AI as they do all the while believing that AI will merely be "a better search than google".
Either way, the spending is of course very large. But this updates me personally away from RSI myself somewhat. It seems we need exponentially more inputs to get the same capability gains.