The Tensor Economics blog covers the economics of producing text from language models at scale. The posts themselves are wonderfully detailed but somewhat overwhelming. I want to provide a summary of their work that might act as a guide. Then I’ll tie in some complimentary data from SemiAnalysis. Note that...
Companies drive AI development today. There's two stories you could tell about the mission of an AI company: AGI: AI labs will stop at nothing short of Artificial General Intelligence. With enough training and iteration AI will develop a general ability to solve any (feasible) task. We can leverage this...
This idea is already implicit in most people's thinking, but I wanted something I could link to. I propose we use the data scaling hypothesis to refer to the claim: > The quality and quantity of data used to train an AI model is by far the most important determinant...
EDIT: the funding proposal I mentioned at the end is out! See here for more. None of this is medical advice. 5 AM on a Saturday and I can’t go back to sleep. It’s not the first time, so I get up to write, I might as well use the...
I’ve avoided talking about AI, mostly because everyone is talking about it. I think the implications of AI scaling were clear a while ago, and I mostly said my piece back then. It’s time to take another crack at it. Scaling laws Scaling laws are a wonderful innovation[1]. With large...
Some code for this post can be found here. Space tethers take the old, defunct space elevator concept and shorten it. Rockets can fly up to a dangling hook in the sky and then climb to a higher orbit. If the tether rotates, it can act like a catapult, providing...
Code for some of the equations and plots can be found here. Charles I. Jones’ paper The A.I. Dilemma: Growth versus Existential Risk tackles the question of whether or not to pursue AI development in the face of existential risk. I found the approach both interesting and accessible and I...