Speak Not of Data Inefficiency
Humans are really bad at comparing themselves to models. Take, for example, Leopold's preschooler graph: Set aside, for a moment, concerns of scaling and RSI, and meditate: what characteristics does GPT-2 share with a preschooler? * Rudimentary control of human language * The ability to count the R's in "strawberry"...
Oct 127