I'm not sure it is different than for humans, honestly. First, I should give a standard disclaimer that different students have different strengths and weaknesses in terms of mathematical problem-solving ability, as well as different aesthetic preferences for what types of problems they like to work on, so any overview like the one I am about to give is necessarily reductive and doesn't capture the full range of opinions on this matter.
As I recall from my own Math olympiad days (and, admittedly, it has been quite a while), Combinatorics problems were generally considered to be the most difficult/tricky/annoying for the majority of contestants. This was because, unlike the vast majority of other fields, Combinatorics problems required a certain set of intuitions that could be described as "combinatorial taste", and solving them required you to understand the particular problem in front of you deeply, on an S1 level. You generally had to "play around" with the set-up for a fair bit, developing a better and better picture of what's going on, before certain insights just clicked in your head and guided you on the path to a solution.[1]
Geometry, by contrast, was much more straightforward.[2] Not in the sense that the problems were necessarily easier from an "objective" standpoint,[3] but because you had an essentially fixed set of techniques and configurations you needed to understand (and mostly memorize) that would solve the vast majority of problems. As Evan Chen has said:
Very roughly, there are three different ways I try to make progress on a geometry problem.
- (I) The standard synthetic techniques; angle chasing, cyclic quadrilaterals, homothety, radical axis / power of a point, etc. My own personal arsenal contains some weapons not known to many contestants as well, most notably inversion, harmonic bundles and quadrilaterals, and spiral similarity / Miquel points.For this part, it’s highly advantageous to be well-versed with “standard” configurations and tricks. To give an extreme example: to solve Iran TST 2009, Problem 9 one essentially needs only recognize two configurations: a lemma about the midpoint of an altitude (2002 G7) and another lemma about the line
(USAJMO 2014/6). Not knowing either of these makes it more difficult to solve the problem synthetically in the time limit. As a reference, Yufei Zhao’s lemmas handout contains a fairly comprehensive list of these configurations.
You also often had the opportunity to turn a geometry problem into an algebra bash, through complex numbers, barycentric coordinates, Cartesian coordinates, trigonometry, etc. These were skills you developed naturally by practice, and in many cases, if you were an experienced and well-prepared contestant, whenever you were given a Geometry problem, you would be able to instantly remember related problems and ideas and figure out the rough plan of how you needed to go about solving the problem. This is completely the opposite of Combinatorics, where you did indeed sometimes have recurring patterns or broad themes,[4] but you mostly had to take each (worthwhile, IMO-level) problem on its own terms, instead of falling back on previously-memorized ideas.[5]
Rather interestingly, this phenomenon (the distinction between the aesthetic of Combinatorics being centered on "deeply understanding and solving one problem" and the rest of the fields as "applying general techniques that expand the understanding of the entirety of Geometry, Algebra, etc.") was also noted at the research level by Timothy Gowers in his excellent essay on "The Two Cultures of Mathematics."[6]
So, from this perspective, I suppose it would make a lot of sense for a rather narrow AI system to be better at areas of olympiad Math that are more easily "systematizable" (such as Algebra and Geometry) and worse at the areas that seem to require a sort of deep understanding and problem-solving (like Combinatorics) that could plausibly get it closer to general intelligence.
I think the standard example to illustrate this is problem 2 of the 2011 IMO (nicknamed the "windmill problem"), which had an average score of 1.5/7, which was roughly half of what the organizers generally aim for (meaning ~ 3/7, as in 2012), mostly because they underestimated how difficult it would be for the contestants to understand the (combinatorial) set-up properly given the time constraints.
And so was Algebra, but to a lesser extent (although contestants working on inequalities, for example, also benefitted from a boatload of books filled to the brim with dozens of techniques and proof methods specifically geared for this).
I'm not sure how such a thing could even be defined, anyway.
If there actually was no structure to Combinatorics problems, then human minds would not be able to predictably perform better at solving them over time as they practiced more and more, which is false.
To be fair, I have heard through the grapevine that this is getting less and less true as the years pass, because trainers are getting better and better at identifying and compiling books/documents, etc., for training specific combinatorial problem-solving techniques.
Of course, there is a lot of nuance here that I am not getting into, and you shouldn't come away with the impression that Combinatorics is all just one-time tricks and the rest of math is just memorizing proofs that worked on previous problems. Reality is far more complex and multi-dimensional than that; there are parts of Combinatorics that work on the basis of deep theories (such as when using generating functions or the probabilistic method or when making connections with Statistical Mechanics, etc.), and also non-Combinatorial results that seem to have arisen from an almost purely individualized assessment of the problem (David Corfield mentions arithmetic progressions among primes as an example).
Google DeepMind reports on a system for solving mathematical problems that allegedly is able to give complete solutions to four of the six problems on the 2024 IMO, putting it near the top of the silver-medal category.
Well, actually, two systems for solving mathematical problems: AlphaProof, which is more general-purpose, and AlphaGeometry, which is specifically for geometry problems. (This is AlphaGeometry 2; they reported earlier this year on a previous version of AlphaGeometry.)
AlphaProof works in the "obvious" way: an LLM generates candidate next steps which are checked using a formal proof-checking system, in this case Lean. One not-so-obvious thing, though: "The training loop was also applied during the contest, reinforcing proofs of self-generated variations of the contest problems until a full solution could be found."
[EDITED to add:] Or maybe it doesn't work in the "obvious" way. As cubefox points out in the comments below, DeepMind's description doesn't explicitly say that it does that, and they're infuriatingly vague about how "AlphaProof generates solution candidates". The previous paragraph at least insinuates that it does it with an LLM, but it's very unclear.
(That last bit is reminiscent of something from the world of computer go: a couple of years ago someone trained a custom version of KataGo specifically to solve the infamous Igo Hatsuyoron problem 120, starting with ordinary KataGo and feeding it training data containing positions reachable from the problem's starting position. They claim to have laid that problem to rest at last.)
AlphaGeometry is similar but uses something specialized for (I think) Euclidean planar geometry problems in place of Lean. The previous version of AlphaGeometry allegedly already performed at gold-medal IMO standard; they don't say anything about whether that version was already able to solve the 2024 IMO problem that was solved using AlphaGeometry 2.
AlphaProof was able to solve questions 1, 2, and 6 on this year's IMO (two algebra, one number theory). It produces Lean-formalized proofs. AlphaGeometry 2 was able to solve question 4 (plane geometry). It produces proofs in its own notation.
The solutions found by the Alpha... systems are at https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/imo-2024-solutions/index.html. (There are links in the top-of-page navbar to solutions to the individual problems.)
(If you're curious about the IMO questions or want to try them yourself before looking at the machine-generated proofs, you can find them -- and those for previous years -- at https://www.imo-official.org/problems.aspx.)
One caveat (note: an earlier version of what I wrote failed to notice this and quite wrongly explicitly claimed something different): "First, the problems were manually translated into formal mathematical language for our systems to understand." It feels to me like it shouldn't be so hard to teach an LLM to convert IMO problems into Lean or whatever, but apparently they aren't doing that yet. [EDITED to add:] But it seems they are doing this to generate training data. Perhaps the fact that they didn't do it for the IMO problem statements themselves reveals something about the limitations of their English-to-Lean translator?
Another caveat: "Our systems solved one problem within minutes and took up to three days to solve the others." Later on they say that AlphaGeometry 2 solved the geometry question within 19 seconds, so I guess that was also the one that was done "within minutes". Three days is a lot longer than human IMO contestants get given, but this feels to me like the sort of thing that will predictably improve pretty rapidly.