Scaling Laws

Applied to My ML Scaling bibliography by gwern at 3mo


Scaling laws graph](


Is it not possible to use images in tags? Or am I just using the wrong syntax?

1plex3moIt is possible, you just paste the image apparently, thanks Yoav Ravid for the tip.

Scaling Laws refer to the observed trend of some machine learning architectures (notably transformers) to scale their performance on predictable power law when given more compute, data, or parameters (model size), assuming they are not bottlenecked on one of the other resources. This has been observed as highly consistent over more than six orders of magnitude.

![Scaling laws graph](

Graph from Scaling Laws for Neural Language Models

Created by leogao at 7mo