x

LESSWRONG
LW

Nick_Greig — LessWrong

Nick_Greig

Nick_Greig

Message

10

Ω

3

1

3y

Nick_Greig

Updates on scaling laws for foundation models from ' Transcending Scaling Laws with 0.1% Extra Compute'

I am not sure if this paper is flying under the radar for many people, but has anyone read Transcending Scaling Laws with 0.1% Extra Compute? If so, how do you think it compares to the scaling laws presented by Deepmind's An empirical analysis of compute-optimal large language model training?...

Nov 18, 2022•15

Message

10 karma

3 alignment forum karma

1 post

Member for 3 years

Updates on scaling laws for foundation models from ' Transcending Scaling Laws with 0.1% Extra Compute'

3y

I am not sure if this paper is flying under the radar for many people, but has anyone read Transcending Scaling Laws with 0.1% Extra Compute? If so, how do you think it compares to the scaling laws presented by Deepmind's An empirical analysis of compute-optimal large language model training? Does it make you rethink the importance of dataset size (again)?

2

15