This post is one part of the sequence Understanding the diffusion of large language models. As context for this post, I strongly recommend reading at least the 5-minute summary of the sequence.
This post lists questions about AI diffusion that I think would be worthy of more research at the time of writing. Some questions serve as direct follow-ups to my research, while others just seem like important questions related to diffusion. I already raised some of these questions throughout this sequence, but this post collects them all so that interested researchers can easily refer back to the questions.
Feel free to reach out to me about these research ideas. I may be able to offer advice, suggest links, and suggest people to talk to. It's possible that I or Rethink Priorities could help connect you with funding to work on these ideas if you're interested and a good fit.
In a previous post I presented proposals to limit access to datasets and proposals to limit access to algorithmic insights. I believe those proposals are probably worth doing, but that belief has a low enough resilience that the next step should be further consideration of whether or not to do these things.
The follow-up questions for the dataset proposals that I think are highest priority are:
The follow-up questions for the algorithmic insight proposals that I think are highest priority are:
As I noted in the section clarifying the scope of this sequence, I have focused on the diffusion mechanisms of replication and incremental research, because they are the most relevant mechanisms in my case studies. But it is worth researching the role of other mechanisms—both historically and in the future. These mechanisms include leak, theft, espionage, and extortion—see the definitions section for explanations of each of these.
Here are questions to ask about each of these mechanisms:
I’d guess about one full-time equivalent month of further research on risks from each of these other mechanisms would be worthwhile for someone in the AGI governance community to do. People with an information security background seem like an especially good fit given the nature of the mechanisms. A history background may also be useful given that there is historical precedent of these mechanisms in domains outside of AI.
The case studies presented here were limited to the domain of language model pretraining. Future work could study cases of diffusion in other domains of AI. This would be useful both to expand the overall amount of empirical data on diffusion, and to make comparisons to my existing case studies. For instance, is there any evidence from other cases that counters the conclusions I drew from my case studies so far?
Some examples of cases that could be studied are:
I think it would be worth someone spending one full-time equivalent week on each of the above case studies. Additionally, I think it would be worth spending one full-time equivalent month to get data on “time to open-source” for many prominent ML breakthroughs or models, and analyze that data. For that research, people with some background in ML would be a good fit. I would also recommend consulting experts in the ML field about what the biggest breakthroughs are, and whether they know of specific open-source versions of implementations and models.
However, I would not recommend undertaking another investigation as broad as the one I have done for GPT-3-like models. I think it would be more productive to focus on questions that have a narrower and more easily isolated scope, one at a time, like many of the questions I have listed in this post.
It’s important to consider the publication practices of emerging AI developers, who could plausibly catch up to current leaders (at least in certain domains). Three examples are Adept, Cohere, and Stability. Based on their first announced AI system, Adept seems to be adopting a closed publication strategy for now. Cohere also appears to be closed about their models—I failed to find any information on their website even about the parameter count of their models in production. This strategy seems to be in their interest, for the sake of protecting commercial IP. And yet Stability seems to have gained a lot from the public release of the “Stable Diffusion” text-to-image model, as it attracted investment and future customers.
How will these emerging developers respond to their own increasing capabilities and revenue, in terms of publication norms? Will Stability be more protective of IP in the future, and/or become more sensitive to misuse concerns? I’m uncertain how to approach these questions productively, but I think it is worth someone thinking about how to approach them and then gathering information on that basis for at least one full-time equivalent week. One idea is simply to ask these developers directly about their strategy and attitudes. Another idea is to look at historical precedents.
I have presented my case for the deployment costs of GPT-3-like models most likely being one order of magnitude less than development costs, even for the largest viable deployment scenarios (see this previous post). However, this had a lot of uncertainty, and I haven’t answered the same question about other domains of AI, and about transformative AI in the future. So the following questions remain.
I have presented my best guess about the relative importance of different inputs to AI development (see this section of a previous post). But I still have a lot of uncertainty about this. Some of the key uncertainties are below.
Here is a list of other questions. The fact that I don’t elaborate as much on these questions here does not mean they are less important. I either spent less time thinking about them, or I don’t think they need as much elaboration.
This research is a project of Rethink Priorities. It was written by Ben Cottier. Thanks to Alexis Carlier, Amanda El-Dakhakhni, Ashwin Acharya, Ben Snodin, Bill Anderson-Samways, Erich Grunewald, Jack Clark, Jaime Sevilla, Jenny Xiao, Lennart Heim, Lewis Ho, Lucy Lim, Luke Muehlhauser, Markus Anderljung, Max Räuker, Micah Musser, Michael Aird, Miles Brundage, Oliver Guest, Onni Arne, Patrick Levermore, Peter Wildeford, Remco Zwetsloot, Renan Araújo, Shaun Ee, Tamay Besiroglu, and Toby Shevlane for helpful feedback. If you like our work, please consider subscribing to our newsletter. You can explore our completed public work here.
For example, stating that your password is more than 20 characters long, or that you have 2-factor authentication in place on an account, just communicates that you have relatively strong security rather than exposing a vulnerability.
See the Abstract of Silver et al. (2017) presenting AlphaGo Zero.
As one comparison, I estimated PaLM’s actual final training run cost (for Google) at about $6 million. One issue with the $35 million estimate is that AlphaGo Zero was trained using TPUs. While TPUs made training much faster (and perhaps much more feasible) than using GPUs, it was also much more expensive than GPUs at the time—see the data in the Footnotes in H. (2020) about the estimate, which lists $6.50/hour for TPU vs. $0.31/hour for GPU.
This is based on Tian et al. (2019) stating “ELF OpenGo is the first open-source Go AI to convincingly demonstrate superhuman performance with a perfect (20:0) record against global top professionals.”
See the Abstract of Hoffmann et al. (2022): “We find that current large language models are significantly undertrained…This also means that \chinchilla uses substantially less compute for fine-tuning and inference, greatly facilitating downstream usage.”
See this draft report by Lennart Heim (requires access). This forecast is lower than the one in CSET’s Lohn and Musser (2022, p. 13) because it uses different (more reliable) trends of compute doubling times and GPU price performance.