In addition to this, Microsoft will exert greater pressure to extract mundane commercial utility from models, compared to pushing forward the frontier. Not sure how much that compensates for the second round of evaporative cooling of the safety-minded.

Microsoft practices "Embrace and extinguish" or "monopolistic copier" as their corporate philosophy. So you can expect them to reproduce a mediocre version of gpt-4 - probably complete with unreliable software and intrusive pro Microsoft ads - and to monopolistically occupy the "niche". Maybe. They are really good at niche defense so they would keep making the model better.

Don't celebrate too early though. Chaos benefits accelerationists. Diversity of strategy. If multiple actors - governments, corporations, investors, startups - simply choose what to do randomly, there is differential utility gain in favor of AI. More AI, stronger AI, uncensored and unrestricted AI. All of these things will give the actors who improve AI more investment and so on in a runaway utility gain. (This is the Fermi paradox argument as well. So long as alien species have a diversity of strategy and the tech base for interstellar travel, the expansionists will inevitably fill the stars with themselves)

This is why one point of view is to say that since other actors are certain to have powerful AGI at their disposal as soon as the compute is available to find it, your best strategy is to be first or at least not to be behind by much.

In the age of sail, if everyone else is strapping cannons on their boats, you better be loading your warships with so many guns the ship barely floats. Asking for an international cannon ban wasn't going to work, the other signatories would claim to honor it and then in the next major naval battle, open up their gun ports.

the sort of lumbering big company where great research or tech is developed by one team and then never reaches anybody else

... except one of our primary threat models is accident risk where the tech itself explodes and the blast wave takes out the light cone. Paraphrasing, the sort of "great tech" that we're worrying about is precisely the tech that would be able to autonomously circumvent this sort of bureaucracy-based causal isolation. So in this one case, it matters comparatively little how bad Microsoft is at deploying its products, compared to how well it can assist their development.

I mean, I can buy that Microsoft is so dysfunctional that just being embedded into it would cripple OpenAI's ability to even do research, but it sounds like Sam Altman is pretty good at what he does. If it's possible to do productive work as part of MS at all, he'd probably manage to make his project do it.

Nicely done! I only come here for the humour these days.

Ummm, the laminated monkey hierarchy is going to determine exactly who launches the first AGI, and therefore who makes the most important call in humanity's history.

If we provide them a solid alignment solution that makes their choice easier, but it's still going to be some particular person's call.