I’m pleased to report that I have no new AI-related crises for you this week. Instead we get to focus on the fun parts, starting with physical constraints on AI development. Dylan Patel explains how power, GPUs, and memory will each be crucial bottlenecks on AI development over the next few years. Turning our attention to AI itself, we'll ask two leading neuroscientists whether AI is likely to become conscious (conclusion: probably yes, or almost certainly not).
AI is doing fascinating things to programmers: for many of us, this moment is simultaneously exhilarating and slightly heartbreaking. We’ll look at one high level overview of how AI is affecting programming, and one deeply personal reflection on that same topic. Programmers aren’t the only ones being disrupted: prinz joins us to argue that while the legal profession will survive AI, the big law firms will not.
If you’re here for the AI, it may not be clear to you why you should listen to a two and a half hour podcast about semiconductors. But this one features Dwarkesh Patel and Dylan Patel and it’s really good—it’s super interesting, but also maps out some of the most important strategic questions that will shape AI over the next few years. A few highlights:
Compute capacity is perhaps the most important factor limiting AI progress right now. That will remain true indefinitely, but it’s more complicated than simply needing more chips. Power, GPUs, and memory will all be critical bottlenecks at different points over the next five years.
Even though GPUs are quickly becoming much more powerful, the value of the work they do is increasing even faster. The counter-intuitive result is that each generation of GPU may increase in value as it moves toward obsolescence (more modern generations will, of course, be even more valuable).
A consequence of AI becoming so valuable is that technologies that compete for the same components (like cell phones and gaming computers) are likely to become more expensive and possibly less capable for a few years.
The US has an enormous compute advantage at the moment, but China will probably overtake us—perhaps sometime between 2030 and 2035. (That significantly complicates the game theory of an international AI pause, incidentally).
Is Elon right that it makes sense to put data centers in space? Yes, but not nearly as soon as he thinks.
It’s a really good podcast—go listen to it (or read the transcript).
Zvi reviews GPT-5.4. This looks like a very substantial upgrade, and it’s getting great reviews. If you use AI heavily and you haven’t played with GPT in a while, now is a good time to give it another try.
Nice: Opus 4.6 and Sonnet 4.6 now have a 1 million token context window. I spent much of this weekend coding and the bigger context window was fantastic.
Andrej Karpathy continues to push the frontier of one-person AI development. His most recent project is autoresearch: an autonomous AI system that makes improvements to his nanochat AI:
This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones.
I love coding in 2026: I’m several times more productive than I’ve ever been before, and it’s absolutely intoxicating. You can have my agentic coding models when you pry them from my cold, dead fingers. But at the same time, I mourn the loss of parts of my craft that just a year ago were important parts of my identity.
Ajeya Cotra shares some very interesting thoughts on METR’s time horizon metric. This piece has received attention because she’s changing her January prediction that the metric will reach 24 hours by the end of this year. Based on recent progress (it’s already reached 12 hours), she’s now predicting 100 hours by the end of the year.
Even more interesting to me is her discussion of how the metric starts to fall apart beyond a certain point. She suggests that almost no tasks really have a one year time horizon: software tasks that would take a human a year to complete are really a collection of multi-day or maybe multi-week tasks that are largely independent.
We’re quickly running out of traditional benchmarks that can usefully measure the capability of frontier models. Where we’re going, there is no map and no speedometer.
One criticism of Claude’s Constitution is “that couldn’t possibly work”. aryaj investigated how well it’s working as part of the MATS program. The results are far from definitive, but very encouraging:
Anthropic has gotten much better at training the model to follow its constitution! Sonnet 4.6 has a 1.9% violation rate, Opus 4.6 is at 2.9%, and Opus 4.5 is at 4.4%.
As a control, Sonnet 4, which did not have special soul doc training, has a ~15.00% violation rate.
Might China be open to an international treaty to pause AI development? In part, that depends on how concerned China is about AI safety, which is complicated. On the one hand, China takes AI safety much more seriously than the US, requiring all AI products to obtain an extensive AI safety certification. On the other hand, “AI safety” is more concerned with ideological correctness and “core socialist values” than existential risk.
ChinaTalk explores the business side of AI safety compliance in China, shedding light on a field I previously knew very little about.
The AI Whistleblower Initiative presents 6 in-depth profiles of whistleblowers at AI companies, exploring the concerns they raised, what impact they had, and what cost they paid.
prinz believes BigLaw will not survive the AI era. He argues that with AI, a senior partner plus a small number of specialists and support staff will be able to do everything a BigLaw firm does today.
This is a likely path for many professions: with AI, the best people in a field can do far more than previously (and get paid accordingly). But the rank and file will find themselves increasingly unemployable.
Dwarkesh wades into the DoW / Anthropic dispute. I don’t agree with everything here, but it’s a really good piece that explores some of the very challenging questions about who gets to make the big decisions in our near future.
Our future civilization will run on AI labor. And as much as the government’s actions here piss me off, in a way I’m glad this episode happened - because it gives us the opportunity to think through some extremely important questions about who this future workforce will be accountable and aligned to, and who gets to determine that.
Are LLMs likely to become conscious as they approach human-level intelligence? That’s a highly contested topic, with lots of strongly held opinions but not a lot of evidence. Even experts on consciousness can’t seem to agree: this week brings us opposing opinions from two well-regarded experts.
Michael Graziano (originator of Attention Schema Theory) tells PRISM that AI consciousness seems likely, and argues that conscious AI might be safer than “zombie AI”.
I’ll publish a longer piece on Wednesday examining Anil’s argument in more detail (sneak preview: I have a lot of respect for him, but in this matter I think he’s overconfident).
Anthropic is moving toward letting employees sell $6b worth of shares. A significant fraction of that is likely to be donated to effective altruism-aligned causes (which would be great) as well as AI safety causes (where it might make a very significant difference).
Open models have struggled to gain widespread adoption: the best models are quite good, but simply can’t compete with the frontier. Nathan Lambert surveys the state of the open model ecosystem and explores where open models are most likely to succeed. I like his idea of open models that are cheap and fast and can be trained for specific tasks, though I’m not sure that will see widespread adoption in the near future.
New from NVIDIA: Nemotron 3 Super is an open model with strong performance and a ton of supporting data and training information. It’s not competitive with the frontier, but Nathan Lambert believes it’s a big deal for the open model world.
Out-of-context reasoning is “when an LLM reaches a conclusion that requires non-trivial reasoning but the reasoning is not present in the context window”. It is sometimes the result of reasoning during the training process, and sometimes (increasingly with large modern models) the result of computation that occurs during a single forward pass. Owain Evans has a short but helpful explainer.
Brain emulation has been making rapid progress. We’re still a very long way from being able to emulate a full human brain, but it now seems plausible that we might be less than a decade away from being able to emulate the brains of fruit flies or other relatively simple organisms.
Asimov Press and Maximilian Schons review what’s currently possible, discuss the technological obstacles that still need to be surmounted, and lay out a roadmap for achieving full emulation of a human brain.
I can confirm that food tastings are a fantastic and low-effort way to create shared experience, not to mention an excellent excuse to eat a lot of good food.
I’m pleased to report that I have no new AI-related crises for you this week. Instead we get to focus on the fun parts, starting with physical constraints on AI development. Dylan Patel explains how power, GPUs, and memory will each be crucial bottlenecks on AI development over the next few years. Turning our attention to AI itself, we'll ask two leading neuroscientists whether AI is likely to become conscious (conclusion: probably yes, or almost certainly not).
AI is doing fascinating things to programmers: for many of us, this moment is simultaneously exhilarating and slightly heartbreaking. We’ll look at one high level overview of how AI is affecting programming, and one deeply personal reflection on that same topic. Programmers aren’t the only ones being disrupted: prinz joins us to argue that while the legal profession will survive AI, the big law firms will not.
Top pick
A deep look at compute constraints
If you’re here for the AI, it may not be clear to you why you should listen to a two and a half hour podcast about semiconductors. But this one features Dwarkesh Patel and Dylan Patel and it’s really good—it’s super interesting, but also maps out some of the most important strategic questions that will shape AI over the next few years. A few highlights:
It’s a really good podcast—go listen to it (or read the transcript).
New releases
GPT-5.4 Is A Substantial Upgrade
Zvi reviews GPT-5.4. This looks like a very substantial upgrade, and it’s getting great reviews. If you use AI heavily and you haven’t played with GPT in a while, now is a good time to give it another try.
1M context in Opus and Sonnet
Nice: Opus 4.6 and Sonnet 4.6 now have a 1 million token context window. I spent much of this weekend coding and the bigger context window was fantastic.
Agents!
Andrej Karpathy’s autoresearch project
Andrej Karpathy continues to push the frontier of one-person AI development. His most recent project is autoresearch: an autonomous AI system that makes improvements to his nanochat AI:
If you want to go deeper, here’s a great annotated version of the prompt.
The End of Computer Programming as We Know It
I love coding in 2026: I’m several times more productive than I’ve ever been before, and it’s absolutely intoxicating. You can have my agentic coding models when you pry them from my cold, dead fingers. But at the same time, I mourn the loss of parts of my craft that just a year ago were important parts of my identity.
This week brings two very different issues exploring how programmers are adapting to agentic coding. Clive Thompson has a carefully researched piece for the NY Times ($), and James Randall has a deeply personal reflection.
Benchmarks and Forecasts
I underestimated AI capabilities (again)
Ajeya Cotra shares some very interesting thoughts on METR’s time horizon metric. This piece has received attention because she’s changing her January prediction that the metric will reach 24 hours by the end of this year. Based on recent progress (it’s already reached 12 hours), she’s now predicting 100 hours by the end of the year.
Even more interesting to me is her discussion of how the metric starts to fall apart beyond a certain point. She suggests that almost no tasks really have a one year time horizon: software tasks that would take a human a year to complete are really a collection of multi-day or maybe multi-week tasks that are largely independent.
We’re quickly running out of traditional benchmarks that can usefully measure the capability of frontier models. Where we’re going, there is no map and no speedometer.
Alignment and interpretability
How well do models follow their constitutions?
One criticism of Claude’s Constitution is “that couldn’t possibly work”. aryaj investigated how well it’s working as part of the MATS program. The results are far from definitive, but very encouraging:
Are we dead yet?
Making Money in Chinese AI Safety
Might China be open to an international treaty to pause AI development? In part, that depends on how concerned China is about AI safety, which is complicated. On the one hand, China takes AI safety much more seriously than the US, requiring all AI products to obtain an extensive AI safety certification. On the other hand, “AI safety” is more concerned with ideological correctness and “core socialist values” than existential risk.
ChinaTalk explores the business side of AI safety compliance in China, shedding light on a field I previously knew very little about.
What Happens When AI Insiders Speak Up?
The AI Whistleblower Initiative presents 6 in-depth profiles of whistleblowers at AI companies, exploring the concerns they raised, what impact they had, and what cost they paid.
Jobs and the economy
Why prinz thinks AI will kill BigLaw
prinz believes BigLaw will not survive the AI era. He argues that with AI, a senior partner plus a small number of specialists and support staff will be able to do everything a BigLaw firm does today.
This is a likely path for many professions: with AI, the best people in a field can do far more than previously (and get paid accordingly). But the rank and file will find themselves increasingly unemployable.
Strategy and politics
I’m glad the Anthropic fight is happening now
Dwarkesh wades into the DoW / Anthropic dispute. I don’t agree with everything here, but it’s a really good piece that explores some of the very challenging questions about who gets to make the big decisions in our near future.
AI psychology
Opposing viewpoints on AI consciousness
Are LLMs likely to become conscious as they approach human-level intelligence? That’s a highly contested topic, with lots of strongly held opinions but not a lot of evidence. Even experts on consciousness can’t seem to agree: this week brings us opposing opinions from two well-regarded experts.
Michael Graziano (originator of Attention Schema Theory) tells PRISM that AI consciousness seems likely, and argues that conscious AI might be safer than “zombie AI”.
In the opposing corner is Anil Seth (previously), with a short video presenting four reasons why he thinks AI consciousness is extremely unlikely.
I’ll publish a longer piece on Wednesday examining Anil’s argument in more detail (sneak preview: I have a lot of respect for him, but in this matter I think he’s overconfident).
Three AI psychology reading lists
If you’re interested in going deeper on AI psychology and welfare, here are three reading lists to get you started.
Robert Long presents an AI Welfare Reading List and a selection of readings on self knowledge and introspection. Both lists look excellent but focus heavily on academic papers.
Avi Parrack and Štěpán Los have put together a Digital Minds quickstart guide that might be more accessible to casual readers.
Industry news
Anthropic employees say they’ll give away billions. Where will it go?
Anthropic is moving toward letting employees sell $6b worth of shares. A significant fraction of that is likely to be donated to effective altruism-aligned causes (which would be great) as well as AI safety causes (where it might make a very significant difference).
Transformer explores where the money might go.
Open models
What comes next with open models
Open models have struggled to gain widespread adoption: the best models are quite good, but simply can’t compete with the frontier. Nathan Lambert surveys the state of the open model ecosystem and explores where open models are most likely to succeed. I like his idea of open models that are cheap and fast and can be trained for specific tasks, though I’m not sure that will see widespread adoption in the near future.
Nemotron 3 Super
New from NVIDIA: Nemotron 3 Super is an open model with strong performance and a ton of supporting data and training information. It’s not competitive with the frontier, but Nathan Lambert believes it’s a big deal for the open model world.
Technical
Out-of-Context Reasoning
Out-of-context reasoning is “when an LLM reaches a conclusion that requires non-trivial reasoning but the reasoning is not present in the context window”. It is sometimes the result of reasoning during the training process, and sometimes (increasingly with large modern models) the result of computation that occurs during a single forward pass. Owain Evans has a short but helpful explainer.
Building Brains on a Computer
Brain emulation has been making rapid progress. We’re still a very long way from being able to emulate a full human brain, but it now seems plausible that we might be less than a decade away from being able to emulate the brains of fruit flies or other relatively simple organisms.
Asimov Press and Maximilian Schons review what’s currently possible, discuss the technological obstacles that still need to be surmounted, and lay out a roadmap for achieving full emulation of a human brain.
Side interests
Food tastings as an underrated source of meaning
I can confirm that food tastings are a fantastic and low-effort way to create shared experience, not to mention an excellent excuse to eat a lot of good food.