LESSWRONG
LW

538
Understanding the diffusion of large language models

Understanding the diffusion of large language models

Jan 16, 2023 by Ben Cottier

How might transformative AI technology—or the means of producing it—spread among companies, states, institutions, and even individuals? What might the impact of that be, and how can we minimize risks in light of that?

I think these are the central questions for the study of AI "diffusion"—the spread of artifacts from AI development among different actors, where artifacts include trained models, datasets, algorithms, and code. Diffusion can occur through a variety of mechanisms—not only open publication and replication, but also theft, the leaking of information, and other means. 

As a step towards understanding and beneficially shaping diffusion for transformative AI, this sequence presents my findings from a project to study the diffusion of recent large language models specifically. The project was undertaken at Rethink Priorities, mostly during a Fellowship with the AI Governance & Strategy team. The core of the project was case studies of nine language models that are similar to OpenAI’s GPT-3 model, including GPT-3 itself. However, this sequence also provides a broader background on AI diffusion, discusses tentative implications that my research has for the governance of transformative AI, and outlines questions for further investigation of AI diffusion more broadly.

26Understanding the diffusion of large language models: summary
Ω
Ben Cottier
3y
Ω
1
4Background for "Understanding the diffusion of large language models"
Ben Cottier
3y
0
12GPT-3-like models are now much easier to access and deploy than to develop
Ben Cottier
3y
3
4The replication and emulation of GPT-3
Ben Cottier
3y
0
4Drivers of large language model diffusion: incremental research, publicity, and cascades
Ben Cottier
3y
0
4Publication decisions for large language models, and their impacts
Ben Cottier
3y
0
7Implications of large language model diffusion for AI governance
Ben Cottier
3y
0
4Questions for further investigation of AI diffusion
Ben Cottier
3y
0
4Conclusion and Bibliography for "Understanding the diffusion of large language models"
Ben Cottier
3y
0