I’ve written 7 blogs for Inkhaven so far. That leaves 23 to go.
If you’ve been annoyed by the sudden influx of daily blog posts, good news! I’ve decided to stop sending all of these to my subscribers. I’ll plan to limit it to the ones I think people will be most interested in.
Today’s blog is a list of possible blog posts. I’m curious which ones people are most keen on having me write! Or also: if you have ideas that aren’t listed here…
Jumping right in:
What is happening right now? What is everyone doing and why? A candid account of the situation from my perspective. I think the situation on the ground is an urgent crisis. Other people’s actions don’t seem to match that reality. What gives?
The societal-scale risks of AI My account of what is at stake and how AI threatens it.
Who has the burden of proof for AI x-risk? A lot of people act like the ones who claim that AI could kill everyone need to provide clear-cut evidence. I disagree. I name names. Philosophy ensues.
Why is AI risk such a hard problem? Two blog posts (at least), focused on technical and non-technical aspects. I disagree with basically everyone else in one way or another, so seems good to get my own account of my views out there.
On Rogue AI vs. “Scheming” Rogue AI systems seek to thwart human control in order to achieve their own objectives. “Scheming” is when AI systems seek to deceive humans who are supposed to be in control in order to achieve their own objectives. What’s the difference, and why does it matter?
Stopping AI is easier than regulating AI. People agree we want more AI regulation. The main reason to support an indefinite pause on AI instead of something milder is because the milder things are harder to enforce, and enforcement is going to be hard enough already.
Alternative solutions to AI risk Probably several blog posts. Going over all of the other solutions proposed to address AI risk and discussing why they are inadequate.
What a good future might look like Suppose we stop AI. Then what? I have thoughts.
Marginal risk is BS Why evaluating AI development and deployment decisions in terms of “marginal risk” is a ridiculous idea.
Post-scarcity is BS Reasons to expect we won’t get some sort of post-scarcity utopia even if we, e.g. “solve alignment”. There are quite a few.
Evals as BS Polishing up the arguments I’ve been making since back when I was at UK AISI for why “Evals” are silly.
If you think a pause would be good, you should say so. You’d think this goes without saying, but a number of people I’ve talked to think it’s not important to say this loudly and clearly and publicly. Wild. But I guess maybe I should spell it out, despite the considerable risk of stating the obvious.
AI won’t stay limited to internal deployment. AI companies are racing, and the fastest way to race involves influencing the outside world directly and aggressively acquiring resources, not keeping your geniuses confined to the datacenter.
Why don’t people seem worried about out-of-distribution generalization? I’ve talked to a number of AI safety researchers who are very engaged with the frontier of AI R&D, and they seem to think that we can test AI well enough that we really only need to worry if it can fool our tests. But we don’t how AIs will act in fundamentally new situations, which they are guaranteed to encounter.
Math vs. physics vs. philosophy mindsets in AI safety I studied math. Does that have something to do with why I’m not satisfied with the hand-wavy way people seem to argue that AI systems are safe?
10+ years of arguing about AI risk I’ve been at this for a long time, longer than almost anyone in machine learning. An account of my personal backstory, and how the attitude of others has changed towards AI risk. Probably multiple blog posts.
A Cambrian explosion of artificial life. Ecosystems of artificial life. AI will increasingly interact with the physical world and be embodied in various ways, and I expect this to all get very chaotic very fast, with AI powering a new form of “life” that evolves rapidly and colonizes the planet and beyond. This is a very different picture from the one most people I encounter seem to have.
Dear AI community… An open letter to the rest of the field of AI stating my differences with them and attitude towards the field.
The societal immune system. An argument for optimism about AI risk! Society seems surprisingly functional, given the abundant opportunities for anti-social behavior.
My experiences with COVID. I wrote an email warning Mila about the pandemic and urging them to stay home. People argued I was being alarmist. A few days later, Mila closed for the lockdown.
Reasons not to trust AI (even if you would trust a human that acts the same way). Sometimes people argue that AI is more trustworthy than humans because of its seemingly aligned behavior. I argue we should have a stronger prior that humans are trustworthy because of shared intellectual
Tool AI wants to become Agent AI: redux Internet celebrity gwern famously argued that “Tool AI” would be outcompeted by AI agents. These days, people don’t view agents and tools as opposites. Time to revisit what gwern got right and wrong in this classic post!
Why I think AI will lead to human extinction and not just concentration of power. Inter-elite competition means even the billionaires get gradually disempowered.
I have also considered writing some blog posts that are not related to AI. Or less related to AI, but I’m sort of trying to make progress towards getting my core views on the topic in writing.
I have also considered writing more response posts to things other AI writers write that I find deeply, offensively wrong, such as:
The “AI as normal technology” guys’ take on AI existential risk
Anton Leicht on AI movement building
Holden Karnofsky on Anthropic’s RSP 3.0 (to be fair, I’ve only skimmed it)
Happy to hear your suggestions for which articles, arguments, or authors, I should engage with!
Thanks for reading The Real AI! Subscribe for free to receive new posts and support my work.
I’ve written 7 blogs for Inkhaven so far. That leaves 23 to go.
If you’ve been annoyed by the sudden influx of daily blog posts, good news! I’ve decided to stop sending all of these to my subscribers. I’ll plan to limit it to the ones I think people will be most interested in.
Today’s blog is a list of possible blog posts. I’m curious which ones people are most keen on having me write! Or also: if you have ideas that aren’t listed here…
Jumping right in:
What is happening right now? What is everyone doing and why?
A candid account of the situation from my perspective. I think the situation on the ground is an urgent crisis. Other people’s actions don’t seem to match that reality. What gives?
The societal-scale risks of AI
My account of what is at stake and how AI threatens it.
Who has the burden of proof for AI x-risk?
A lot of people act like the ones who claim that AI could kill everyone need to provide clear-cut evidence. I disagree. I name names. Philosophy ensues.
Why is AI risk such a hard problem?
Two blog posts (at least), focused on technical and non-technical aspects. I disagree with basically everyone else in one way or another, so seems good to get my own account of my views out there.
On Rogue AI vs. “Scheming”
Rogue AI systems seek to thwart human control in order to achieve their own objectives. “Scheming” is when AI systems seek to deceive humans who are supposed to be in control in order to achieve their own objectives. What’s the difference, and why does it matter?
Stopping AI is easier than regulating AI.
People agree we want more AI regulation. The main reason to support an indefinite pause on AI instead of something milder is because the milder things are harder to enforce, and enforcement is going to be hard enough already.
Alternative solutions to AI risk
Probably several blog posts. Going over all of the other solutions proposed to address AI risk and discussing why they are inadequate.
What a good future might look like
Suppose we stop AI. Then what? I have thoughts.
Marginal risk is BS
Why evaluating AI development and deployment decisions in terms of “marginal risk” is a ridiculous idea.
Post-scarcity is BS
Reasons to expect we won’t get some sort of post-scarcity utopia even if we, e.g. “solve alignment”. There are quite a few.
Evals as BS
Polishing up the arguments I’ve been making since back when I was at UK AISI for why “Evals” are silly.
If you think a pause would be good, you should say so.
You’d think this goes without saying, but a number of people I’ve talked to think it’s not important to say this loudly and clearly and publicly. Wild. But I guess maybe I should spell it out, despite the considerable risk of stating the obvious.
AI won’t stay limited to internal deployment.
AI companies are racing, and the fastest way to race involves influencing the outside world directly and aggressively acquiring resources, not keeping your geniuses confined to the datacenter.
Why don’t people seem worried about out-of-distribution generalization?
I’ve talked to a number of AI safety researchers who are very engaged with the frontier of AI R&D, and they seem to think that we can test AI well enough that we really only need to worry if it can fool our tests. But we don’t how AIs will act in fundamentally new situations, which they are guaranteed to encounter.
Math vs. physics vs. philosophy mindsets in AI safety
I studied math. Does that have something to do with why I’m not satisfied with the hand-wavy way people seem to argue that AI systems are safe?
10+ years of arguing about AI risk
I’ve been at this for a long time, longer than almost anyone in machine learning. An account of my personal backstory, and how the attitude of others has changed towards AI risk. Probably multiple blog posts.
A Cambrian explosion of artificial life. Ecosystems of artificial life.
AI will increasingly interact with the physical world and be embodied in various ways, and I expect this to all get very chaotic very fast, with AI powering a new form of “life” that evolves rapidly and colonizes the planet and beyond. This is a very different picture from the one most people I encounter seem to have.
Dear AI community…
An open letter to the rest of the field of AI stating my differences with them and attitude towards the field.
The societal immune system.
An argument for optimism about AI risk! Society seems surprisingly functional, given the abundant opportunities for anti-social behavior.
My experiences with COVID.
I wrote an email warning Mila about the pandemic and urging them to stay home. People argued I was being alarmist. A few days later, Mila closed for the lockdown.
Reasons not to trust AI (even if you would trust a human that acts the same way).
Sometimes people argue that AI is more trustworthy than humans because of its seemingly aligned behavior. I argue we should have a stronger prior that humans are trustworthy because of shared intellectual
Tool AI wants to become Agent AI: redux
Internet celebrity gwern famously argued that “Tool AI” would be outcompeted by AI agents. These days, people don’t view agents and tools as opposites. Time to revisit what gwern got right and wrong in this classic post!
Why I think AI will lead to human extinction and not just concentration of power.
Inter-elite competition means even the billionaires get gradually disempowered.
I have also considered writing some blog posts that are not related to AI. Or less related to AI, but I’m sort of trying to make progress towards getting my core views on the topic in writing.
I have also considered writing more response posts to things other AI writers write that I find deeply, offensively wrong, such as:
The “AI as normal technology” guys’ take on AI existential risk
Anton Leicht on AI movement building
Holden Karnofsky on Anthropic’s RSP 3.0 (to be fair, I’ve only skimmed it)
Happy to hear your suggestions for which articles, arguments, or authors, I should engage with!
Thanks for reading The Real AI! Subscribe for free to receive new posts and support my work.