I am surprised that no one has pointed out the distinct 'kink' in many of these scaling curves, potentially suggests that even larger models would reverse the inverse scaling trend.
That would particularly interesting since it might imply certain competing dynamics in the models capabilities.
Wonder if the organizers had any thoughts?
field sure moves fast...
https://arxiv.org/abs/2211.02011