June Jimenez's Shortform

June Jimenez

Beliefs I have much less credence in than at the start of this year:

- AI safety is a weird topic, and communication of catastrophic risks is difficult or infeasible.

- Technical alignment research is the main way and best way of contributing to making the impacts of AI go better.

- A bilateral deal with China is unlikely.

- Alignment is a well-understood abstraction.

- If alignment is not solved, outcomes are guaranteed to be bad for humans. (To be clear, I still think they are in expectation quite bad and in probability very likely to be bad for humans, but my intuitions were based off of "eventually developed" approximate expected utility maximizers, which is a pretty bad model for LLMs and not a particularly accurate model for what I think of as the minimal near-term system that can trigger explosive RSI.)

- Without regulatory oversight, timelines to general AI systems better than or equal to top humans at most tasks are uncertain, but have a lot of (say, >40%) probability mass beyond 2030.

Overall, I think I see AI 2027 more as a description of what would happen with unrestricted development than a sketch of a prediction whose modal-ish outcomes are very likely to happen. In general, governance work seems to me to have ramped up impressively and I think there's real appetite for Doing Stuff About The AI Thing that there wasn't at the start of the year and was unclear would happen before. I also think AI 2027 is and continues to be more right in terms of the pace of AI progress than virtually all of the community in conversation with it, and more right than even the AIFP's later predictions which have shifted to be a little bit later. I'm glad they have explicitly clarified that their forecast is conditional on going "as fast as technically possible"; I think it's important of them to highlight that it is not necessary or inevitable to go as fast as technically feasible, but my modal and possibly median "technically feasible" timelines are as short as AI 2027's or shotter.

I'm more scared of where it seems we're headed by default and much more hopeful about our ability to change that default trajectory.

June Jimenez14h41

Noosphere8912h70

Definitely agree with the first, fifth beliefs and sixth beliefs, definitely disagree with you about the third belief, partially agree with the second belief, and am much more uncertain than you about the fourth belief.

I don't buy US-China collaboration is all that possible, at least not yet because of reasons like this as well as the fact that export controls are now biting China hard, but I do believe the US government is waking up and will do something about AI in 2027, but what that things is remains to be seen. And this is likely to provide most of th... (read more)