This is a special post for quick takes by Cole Wyeth. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
This has been going on for months; on the bullish side (for ai progress, not human survival) this means some form of self-improvement is well behind the capability frontier. On the bearish side, we may not expect a further speed up on the log scale (since it’s already factored in to some calculations).
I did not expect this degree of progress so soon; I am now much less certain about the limits of LLMs and less prepared to dismiss very short timelines.
With that said… the problems that it has solved do seem to be somewhat exhaustive search flavored. For instance it apparently solved an open math problem, but this involved arranging a bunch of spheres. I’m not sure to what degree LLM insight was required beyond just throwing a massive amount of compute at trying possibilities. The self-improvements GDM reports are similar - like faster matrix multiplication in I think the 4x4 case. I do not know enough about these areas to judge whether AI is essential here or whether a vigorous proof search would work. At the very least, the system does seem to specialize in problems with highly verifiable solutions. I am convinced, but not completely convinced.
Also, for the last couple of months whenever I’ve asked why LLMs haven’t produced novel insights, I’ve often gotten the response “no one is just letting them run long enough to try.” Apparently GDM did try it (as I expected) and it seems to have worked somewhat well (as I did not expect).
Heads up: I am not an AI researcher or even an academic, just someone who keeps up with AI
But I do have quick thoughts as well;
Kernel optimization (which they claim is what resulted in the 1% decrease in training time) is something we know AI models are great at (see RE-Bench and the multiple arXiv papers on the matter, including from DeepSeek).
It seems to me like AlphaEvolve is more-or-less an improvement over previous models that also claimed to make novel algorithmic and mathematical discoveries (FunSearch, AlphaTensor) notably by using better bas... (read more)