All progress from the past millennium gets destroyed in a global disaster. But we have one shot at salvation. We saved the trillions of weights of a superintelligent model that can guide us to abundance: what is the quickest path to accessing that oracle of intelligence again? — LessWrong
Obviously humans doing arithmetic is too slow. What kind of computers do we build and how much compute do we really need? Progress will compound as we access more of the intelligence.
A mature science of mechanistic interpretability would be a huge speed multiplier.
It's not possible for all and only progress since ~1100CE to be wiped out. Some knowledge will be durable - possibly things like the germ theory of disease and similar. Some artifacts will be durable - evidence of cities and such. Raw materials may be less accessible (easy coal/mineral access, old-growth forests, etc.) or may be MORE accessible if they're recoverable from the artifacts.
The weights of a frontier model are a LONG way from useful - possibly long enough that however they're recorded doesn't last long enough to use them.
This is glossing over the fact that if you can't fit the weights into storage you're not serving the model, ever. It's an all or nothing thing, trillions of parameters is going to be in the terabyte+ range so you're looking at late eighties/early nineties (optimistically) level of tech to conceivably serve a single model, extremely slowly, off the footprint of an entire data center - ONE instance. At best you get to speed through what, forty years? Otherwise you're replaying the first 960 1:1.
Obviously humans doing arithmetic is too slow. What kind of computers do we build and how much compute do we really need? Progress will compound as we access more of the intelligence.
A mature science of mechanistic interpretability would be a huge speed multiplier.