It looks like ARC-AGI-3 is giving us early evidence on being able to use latest AI models to strongly accelerate AI research. The ARC-AGI-3 was running since late March rather sluggishly on the main track. Even at early September, the top score was mostik.ai with 7.51, and it was the clear leader. Since GPT-6 Astra became available, the scores started to climb rapidly, on Sep 24 the top score was 19.40, and now the top score is 27.29:
https://www.kaggle.com/competitions/arc-prize-2026-arc-agi-3/leaderboard
EDIT Sep 29: top score 45.33, second place 36.73 (people have both GPT-6 Astra and Opus 5.5 to assist in AI research now).
Sep 30: 50.65; Oct 3-Oct 8: sitting at 55.89, slow streak; Oct 13: top score 68.91, second place 67.49; Oct 27: 75.56.
I have also thought that it was scored really unfairly, and I think it's very impressive that there is really remarkable progress despite that.
Yet we should not mix the internal track (no internet connection, so no large LLMs can be used directly) and the external track (where Astra has officially scored 99.95% in a good harness).
The external track is a benchmark for LLM abilities, but the internal track, where serious LLMs are not available directly during test runs, is a benchmark for the ability to do AI research.