Hans Gundlach

Graph of NanoGPT progress over time as a fraction of initial training time at the small and medium scale competition tracks. Acknowledgements Thanks to Tom Cunningham's questions for inspiring this post. I also want to acknowledge Parker Whitfill's work on NanoGPT and FLOP vs non-FLOP progress, and Anson Ho's and...

by Hans Gundlach, Jayson Lynch, Matthias Mertens, Neil Thompson (All researchers are part of MIT Futuretech) Epistemic Status: This is a work in progress, and we would appreciate criticism as well as suggestions in all aspects of this work. TLDR: We estimate that general inference efficiency increases 3-10x per year....