How much should we update toward short timelines and hard treaty enforcement because of Flapping Airplanes' continued fundraising?
Flapping Airplanes, a neolab led by twentysomethings with a pitch of "we'll invent AI algorithms that are many OOMs more sample-efficient" raised at a valuation of $1.5B in January. They are now in talks to raise at $5B, from both the $1.5B-round lead (Index Ventures, a prominent generalist firm) and a well-regarded deeptech firm (Lux Capital). (COI note: the latter invested in my company.)
A naive interpretation is: insiders have increased their estimation of FA's success (in creating vastly more sample-efficient AI) 3.3x. My best guess is that this would be justified (explicitly) by "they've hired more smart people" and "they claim technical progress," with perhaps a dash of "I would have invested earlier if they'd let me." Any technical progress is, from what Fable and I can find out, more or less secret. (Everything here is a mix of public info and reliable anonymous sources.)
Reasons to temper that naive interpretation
Things I could learn that would untemper this
How much should it be tempered?
An Mx increase in Flappy's odds of success implies an Nx increase in the odds of our getting OOMs-more-sample-efficient AI in the next (say) 10 years, with N < M. I think strongly that we should believe 1 < M < 3.3 and 1 < N, but I'm not sure what to set as M, let alone N.
Directional implications
From a brutish, scaling-oriented perspective: many OOMs of sample efficiency should (holding compute equal) catapult us forward in effective scaling. Moreover, it makes it more likely, later on, that a single "rogue actor" could develop comprehensively superhuman machine intelligence without a large GPU footprint, making treaties harder to enforce.
I see AI research right now as much more compute-bottlenecked than data-bottlenecked. Yes, I know they trade off against each other, but my impression–––correct me if I'm wrong!–––is that current LLMs are being trained with the optimal (for LLMs) quantity of data to minimize usage of compute. Yes, they're pouring lots of money into higher-quality data, but it's pennies next to the money spent on compute.
From this perspective, then, it has the same effect as some quantum of "algorithmic improvement." And depending on how the data and compute trade off against each other in this fancy new architecture (I haven't run the numbers on how e.g. a 1,000,000x sample-efficiency improvement would play out with Chinchilla laws, because I ought to be working right now, but someone should.)
From a more qualitative perspective: Sample efficiency is a commonly-cited difference between humans/AGI and current AIs! If AIs get continual learning, it will need to be sample-efficient to be truly transformative. This should not only pull forward timelines, but also shorten the gap between "the last AI humans built" and "the first comprehensively superhuman AI." For folks who are less alignment-doomer than I am may draw some encouragement from the second, depending on your theories.
So. What's it all mean? Am I just being distracted by a shiny object? Or is this a big deal?
Thanks for the analysis! I saw a video of Carmack last year, in which he said (roughly) "when I got into AI, my thought was to improve the hardware performance, since that's what I'm best at... but it's already so good that I quit trying, and went to the software end".
It's certainly not impossible that these folks have found something Carmack didn't, but your analysis makes me lean towards this mostly being hype.