This is a linkpost for https://vale.rocks/posts/ai-model-history-is-being-lost
I agree this is really important - particularly because I think many of the theoretical arguments for expecting misalignment provide empirical comparative hypotheses. Being able to look at semi-independent replicates of behaviour relies on old models being available. I don't know the best way forward because I doubt any frontier lab would release old models under a CC license - maybe some kind of centralised charitable foundation.
I fear that we may lose – and are already losing – the modern history of AI, particularly with regard to the ability to use models that are/were only available in hosted capacities. As AI models are retired and become inaccessible, we’re losing crucial historical context about AI development that we will be unable to go back and reference in the future.
This essay examines the disappearance due to discontinuation of proprietary hosted AI models and the implications of that on research, accountability, and overall historical documentation and preservation.
I fear this is a very under-analysed topic, with New methods for deprecating artificial intelligence systems will preserve history and facilitate research (Johnson T, Obradovich N. 2024) being the only notable discussion I have found.
Continues in post...