This is a special post for quick takes by Haoxing Du. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Apparently this is not very far from how things looked like 2 years ago. KataGo's biggest models are less than 1 GB in size. Transformer support was just added in the latest major release, stating that transformers are "generally much stronger given the same compute cost" compared to the current model architecture. So probably a correctly trained 1B param dense transformer is stronger.
The biggest LLMs of 2026 (Mythos 5, probably Astra) likely have about 1T active params (and 10T total). It's a miracle of narrow AI that KataGo is still so strong.