This is a special post for quick takes by anaguma. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I'm a big fan of OpenAI investing in video generation like Sora 2. Video can consume an infinite amount of compute, which otherwise might go to more risky capabilities research.
My pet AGI strategy, as a 12 year old in ~2018, was to build sufficiently advanced general world models (from YT videos etc.), then train an RL policy on said world model (to then do stuff in the actual world).
A steelman of 12-year-old me would point out that video modeling has much better inductive biases than language modeling for robotics and other physical (and maybe generally agentic) tasks, though language modeling fundamentally is a better task for teaching machines language (duh!) and reasoning (mathematical proofs aren't physical objects, nor encoded in the laws of physics).
OpenAI's Sora models (and also DeepMind's Genie and similar) very much seems like a backup investment in this type of AGI (or at least transformative narrow robotics AI), so I don't think this is good for reducing OpenAI's funding (robots would be a very profitable product class), nor influence (obv. a social network gives a lot of influence, to e.g. prevent an AI pause or to move the public towards pro-AGI views).
In any scenario, Sora 2 seems to me as a net-negative activity for AI safety:
* if LLMs are the way to AGI (which I believe is the case), then we will probably die, but with a more socially influential OpenAI that potentially has robots (than if Sora 2 didn't exist); the power OpenAI would have in this scenario to prevent an AI pause seems to outweigh the slowdown that would be caused by the marginal amounts of compute Sora 2 uses
* if LLMs aren't the way to AGI (unlikely), but world modeling based on videos is (also unlikely), then Sora 2 is very bad - you would want OpenAI to train more LLMs and not invest in world models which lead to unaligned AGI/ASI.
* if neither LLMs or world modeling is the way to AGI (also unlikely), then OpenAI probably isn't using any compute to do 'actual' AGI research (what else do they do?); so Sora 2 wouldn't be affecting the progress of AGI, but it would be increasing the influence of OpenAI; and having highly influential AI companies i
I think that the path to AGI involves LLMs/automated ML research, and the first order effects of diverting compute away from this still seem large. I think OpenAI is bottlenecked more by a lack of compute (and Nvidia release cycles), than by additional funding from robotics. And I hope I'm wrong, but I think the pause movement won't be large enough to make a difference. The main benefit in my view comes if it's a close race with Anthropic, where I think slowing OpenAI down seems net positive and decreases the chances we die by... (read more)