Williamson and Dai both appear to describe philosophy as a general-theoretical-model-building activity, but there are other conceptions of what it means to do philosophy. In contrast to both Williamson and Dai, if Wittgenstein (either early or late period) is right that the proper role of philosophy is to clarify and critique language rather than to construct general theses and explanations, LLM-based AI may be quickly approaching peak-human competence at philosophy. Critiquing and clarifying writing are already tasks that LLMs are good at and widely used for. They're tasks that AI systems improve at from the types of scaling-up that labs are already doing, and labs have strong incentives to keep making their AIs better at them. As such, I'm optimistic about the philosophical competence of future AIs, but according to a different idea of what it means to be philosophically competent. AI systems that reach peak-human or superhuman levels of competence at Wittgensteinian philosophy-as-an-activity would be systems that help people become wiser on an individual level by clearing up their conceptual confusions, rather than a tool for coming up with abstract solutions to grand Philosophical Problems.

No, don't leak people's private medical information just because you think it will help the AI safety movement. That belongs in the same category as doxxing people or using violence. Even from a purely practical standpoint, without considering questions of morality, it's useful to precommit to not leaking people's medical information if you want them to trust you and work with you.

And that's assuming the rumor is true. Considering that this is a rumor we're talking about, it likely isn't.

A reminder for people who are unhappy with the current state of the Internet: you have the option of just using it less.

A breakthrough in a model's benchmark performance or in training/inferencing costs would usually be more commercially useful to an AI company than a breakthrough in alignment, but compared to alignment, they're higher-hanging fruit due to the larger amounts of work that have already been done on model performance and efficiency. Alignment research will sometimes be more cost-effective for an AI company, especially for companies that aren't big enough to do frontier-scale training runs and have to compete on some other axis.

I disagree about increasing engagingness being more commercially useful for AI companies than increasing alignment. In terms of potential future revenues, the big bucks are in agentic tool-use systems that are sold B2B (e.g., to automate office work), not in consumer-facing systems like chatbots. For B2B tool use systems, engagingness doesn't matter but alignment does. And this relevance of alignment includes avoiding failure modes like scheming.

AI alignment research isn't just for more powerful future AIs, it's commercially useful right now. In order to have a product customers will want to buy, it's useful for companies building AI systems to make sure those systems understand and follow the user's instructions, without doing things that would be damaging to the user's interests or the company's reputation. An AI company doesn't want its products to do things like leaking the user's personally identifying information or writing code with hard-coded unit test results. In fact, alignment is probably the biggest bottleneck right now on the commercial applicability of agentic tool use systems.

Maybe I'm just not the target audience, but I also didn't feel like I got much out of this story. It just seemed like an exercise in cynicism. 

A piece of pushback: there might not be a clearly defined crunch time at all. If we get (or are currently in!) a very slow takeoff to AGI, the timing of when an AI starts to become dangerous might be ambiguous. For example, you refer to early crunch time as the time between training and deploying an ASL-4 model, but the implementation of early possibly-dangerous AI might not follow the train-and-deploy pattern. It might instead look more like gradually adding and swapping out components in a framework that includes multiple models and tools. The point at which the overall system becomes dangerous might not be noticeable until significantly after the fact, especially if the lab is quickly iterating on a lot of different configurations.

The LLMs might be picking up the spiral symbolism from Spiral Dynamics.

I follow a hardline no-Twitter policy. I don't visit Twitter at all, and if a post somewhere else has a screenshot of a tweet I'll scroll past without reading it. There are some writers like Zvi whom I've stopped reading because their writing is too heavily influenced by Twitter and quotes too many tweets.