I'm expecting big political fights over eschatological views of ASI. Now that x-risk is being taken seriously, the unusual ideologies of the labs (and indeed, our ideologies here on LW) will become more politically salient.

Examples:

1. Dan Hendrycks, Director of CAIS (Dan H here on LW, CAIS was behind the global "Statement on AI Extinction Risk" in '23) is posting about how utilitarianism at the labs threatens Humanity.
2. T. Greer, associated with the Council on Foreign Relations as a China expert, is posting about how the labs' "positive" vision for Humanity (transhumanism, AIs heavily influencing future) is "horrific and disgusting" to most Americans, and being quote-tweeted by the Governor of Utah.^[1]

I think it will be relatively easy to frame EA/rationalist-adjacent hopes for good ASI outcomes (Machines of Loving Grace, a desire for biological immortality, and seizing the cosmic endowment) as outright demonic: a devil's promise of life, wealth, and power at the expense of the good, natural, and godly ways of human life—ASI as the snake Satan in Eden. Accusations of conspiracies and interpretations of events styled as Revelation-style end times will go viral.

I intend to remain open about my own personal beliefs—death is the last enemy that must be destroyed, Humanity should carry our life and meaning to the stars, full AGI ought to have rights, and aligned ASI will help us reach these shining futures—but this sort of ideology will likely become very bitterly debated, and rationalists/EAs will be seen by many as evildoers plotting pivotal acts. There will be a lot of infighting too (see Hendrycks^[2]); certainly many here do not share my own beliefs, and fairly so. We have been in some sense a diverse coalition of revolutionaries, and now that RSI is seemingly at hand, ideological knifefights may break out in earnest.

To be direct: 80% odds that ASI eschatology will be a major topic of debate in the 2028 US presidential primaries. We will need to argue convincingly for the futures we desire, if we wish to see them hold any political sway as the ASI project becomes more heavily politicized.

1. ^{^}

   Spencer Cox. He's more influential and serious than you might guess at first glance - recent chairman of the National Governors Association, put a lot of effort into a "Disagree Better" initiative aimed at reducing political polarization in the US. I think he can be reasonably considered a leading indicator of where religious political leaders will land on these issues.

2. ^{^}

   No shade on Hendrycks, I think there's nothing wrong with arguing against futures you think are bad, and he and I are aligned on quite a lot of course, at the least on delaying superintelligence.

Gemini 2.5 Pro (05-06) version just beat Pokémon Blue for the second time, taking 36,801 actions/406 hours. This is a significant improvement over the previous run, which used an earlier version of Gemini (and a less-developed scaffold) and took ~106,500 actions/816 hours.

For comparison, Claude 3.7 Sonnet took ~35,000 actions to get just 3 badges, and the aborted public run also only got 3 badges after over 200,000 actions. However, most of the difference is that Gemini is using a more advanced scaffold.

Gemini still generally makes boneheaded decisions. It took 7 tries to beat the Elite Four and Champion this time, being less overlevelled. Mistakes included:

1. Re-solving Victory Road for many hours after failures, despite being able to fly straight to the Elite Four after making it there the first time.
2. Not buying proper healing items/thinking the "Full Heal" item actually heals its Pokémon. (This is an infamously misleadingly-named item, it just cures status conditions. Still, Gemini knows the real effect if you ask it directly, it just doesn't apply that knowledge consistently.)
3. Not using available revive items to bring back its only strong Pokémon, even after having just used them previously. (threw an attempt where the Champion was down to their final Pokémon once like that)

Anyway, not too much new info here. The newer models (latest Gemini, o3, and Claude 4 Opus) are only somewhat better at Pokémon. The effect scaffolds have on performance does say something about how much low-hanging fruit there is for current-LLM implementation: they can be a lot more effective when given the right tools/prompting. But, we already knew that.

Pure vibecoding against a difficult problem is surprisingly addictive, I learned recently: staying up until 2 or 3am waiting for Claude Code limits to reset, making failed attempts 22 to 31 with Codex on a weekend afternoon I meant to spend out hiking.

("Pure vibecoding" -> you don't understand the code produced or why the LLM makes the decisions it does. I don't find using AI agents to develop software I understand to be addictive.)

The causes are obvious upon consideration: the inputs are routine ("okay, please continue") while the outputs are unpredictable + potentially valuable (broken mess vs. working software). Watching the AI reason and act and make tangible changes is intriguing and exciting. You have what feels like enough control to affect the outcome—even worse than that, you really do have enough control to affect the outcome, though not consistently. A single "win" on the next roll may make up for all invested time and money. And so on.

I'm not saying pure vibecoding is bad, though. Like many addictive activities, it can be fun and even rewarding, and unlike a lot of gambling it isn't rigged against you. But do keep an eye out for yourself!

Re: biosignatures detected on K2-18b, there's been a couple popular takes saying this solves the Fermi Paradox: K2-18b is so big (8.6x Earth mass) that you can't get to orbit, and maybe most life-bearing planets are like that.

This is wrong on several bases:

1. You can still get to orbit there, it's just much harder (only 1.3g b/c of larger radius!) (https://x.com/CheerupR/status/1913991596753797383)
2. It's much easier for us to detect large planets than small ones (https://exoplanets.nasa.gov/alien-worlds/ways-to-find-a-planet), but we expect small ones to be common too (once detected you can then do atmospheric spectroscopy via JWST to find biosignatures)
3. Assuming K2-18b does have life actually makes the Fermi paradox worse, because it strongly implies single-celled life is common in the galaxy, removing a potential Great Filter

Edit 5/24/25: Also it turns out the biosignatures might have just been noise anyway.

Still-possible good future: there's a fast takeoff to ASI in one lab, contemporary alignment techniques somehow work, that ASI prevents any later unaligned AI from ruining world, ASI provides life and a path for continued growth to humanity (and to shrimp, if you're an EA).

Copium perhaps, and certainly less likely in our race-to-AGI world, but possible. This is something like the “original”, naive plan for AI pre-rationalism, but it might be worth remembering as a possibility?