“The resources used to train the model can be repurposed to run millions of instances of it (this matches projected cluster sizes by ~2027), and the model can absorb information and generate actions at roughly 10x-100x human speed. … We could summarize this as a ‘country of geniuses in a datacenter’.”

Dario Amodei, CEO of Anthropic, Machines of Loving Grace

“Let’s say each copy of GPT-4 is producing 10 words per second. It turns out they would be able to run something like 300,000 copies of GPT-4 in parallel. And by the time they are training GPT-5 it will be a more extreme situation where just using the computer chips they used to train GPT-5, using them to kind of run copies of GPT-5 in parallel, you know, again, each producing 10 words per second, they’d be able to run 3 million copies of GPT-5 in parallel. And for GPT-6, it’ll just increase again, there’ll be another factor of 10 at play, and so it’ll be 30 million copies in parallel.”

Tom Davidson, researcher at OpenPhil, Future of Life Institute interview

“Once we get to AGI, we won’t just have one AGI. … given inference GPU fleets by then, we’ll likely be able to run many millions of them (perhaps 100 million human-equivalents, and soon after at 10x+ human speed).”

Leopold Aschenbrenner, Situational Awareness pg. 47

Table of Contents

Section I - The Question

Section II - The Scenario

Section III - Existing Estimates

Section IV - Compute

Section VI - Inference

Section V - Human Equivalents

Section VII - The Estimates

Method 1: Total training to inference per token ratio

Method 2: Flat inference costs

Method 3: Human brain equivalent

Method 4: Chip capabilities

Method 5: Adjusting for capabilities per token

Section VIII - Implications

Acknowledgements

Section I - The Question

What will the initial AGI population be?

Artificial intelligence (AI) systems have become significantly more capable and general in the last decade, especially since the launch of ChatGPT in December 2022. Many people believe that the technological trajectory of AI will lead to the advent of artificial general intelligence (AGI), an AI system that can autonomously do virtually anything a human professional can do. Leading AI scientists, like Geoffrey Hinton, Yoshua Bengio, and Shane Legg, are publicly raising the alarm that such a system is incoming. There are several AI enterprises premised on the business model of creating AGI (Anthropic, OpenAI, ...

Posts

Wikitag Contributions

Comments