Even if the AGI were 10 times as expensive and only about as capable on average as a median human (with bursts of superhuman ability like how gpt-3 is way faster and makes few spelling or punctuation errors), there is value in just knowing it is possible. I would expect enormous investments in all of the support infrastructure. After all of you know it is possible and will scale it's a matter of national survival. You cannot afford to be a skeptic when you see the actual flash and mushroom cloud. (Referring to how the los Alamos test obviously converted plenty of skeptics)
Another Manhattan project comparison : wouldn't more damage had been done to japanese cities if the project budget were spent on additional b-29s and ordinance? Two fission devices that only core 2 cities is probably not really enough damage to "justify" the project cost.
GPT-3 is slightly too expensive for many of the use-cases that I am interested in. This problem is made even worse by the fact that one of the basic techniques I normally use in procedural generation is "generate 100 of something and then pick the best one".
It's worth noting here that in a sense, GPT-3 isn't expensive enough if you are trading so much compute to get the necessary quality. You might well be better off with a GPT-4 which cost 10x as much. This is because the best sample out of 100 is only a bit better than the best out of 50, or the best out of 10, or the average sample, but generating 100 samples costs 100x more. If GPT-4 cost up to 100x more to run, then it might still be a win.
Particularly if you include the cost of screening 100 samples and how many workflows that eliminates... Many absolute technical metrics have hard to understand nonlinear translations to enduser utility. Below a certain apparently arbitrary % as defined by accuracy or word error rate or perplexity or whatever, a tool may be effectively useless; and then as soon as it crests it, suddenly it becomes useful for ordinary people. (Speech transcription & machine translation are two examples where I've noticed this.) It could be worth paying much more if it gets you to a level of reliability or quality where you can use it by default, or without supervision, or for entirely new tasks.
I think is a valid point, however in the Artbreeder use-case, generating 100 of something is actually part of the utility, since looking over a bunch of variants and deciding which one I like best is part of the process.
Abstractly, when exploring a high-dimensional space (pictures of cats), it might be more useful to have a lot of different directions to choose from than 2 "much better" directions if the objective function is an external black-box because it allows the black box to transmit "more bits of information" at each step.
Which is the right choice depends on how well we think theoretically it is possible for the Generator to model the black-box utility function. In the case of Artbreeder, each user has a highly individualized utility function whereas the site can at best optimize for "pictures people generally like".
In the particular use-case for GPT-3 I have in mind (generating funny skits), I do think there is in fact "room for improvement" even before attempting to accommodate for the fact that different people have different senses of humor. So in that sense I would prefer a more-expensive GPT-4.
a Senior Software Engineer at Google makes somewhere around $250k/year
This number from Glassdoor omits the $128k/yr average stock compensation, and while you might round startup equity to zero Google stock seems reliably valuable. levels.fyi is a better source for tech total compensation.
Generalizing cousin_it's idea about ems to not-overly-superintelligent AGIs, it's more valuable to run fewer AGIs faster than to run a larger number of AGIs at human speeds. This way AGI-as-a-service might remain in the realm of giant cheesecakes even assuming very slow takeoff.
Having spent quite a bit of time with GPT-3, my feelings can be expressed as:
This is really awesome, but it would be even better if it didn't cost $0.06 per character.
GPT-3 is slightly too expensive for many of the use-cases that I am interested in. This problem is made even worse by the fact that one of the basic techniques I normally use in procedural generation is "generate 100 of something and then pick the best one".
This is actually a noticeable problem with Deep Learning generally in the present moment. Tools like AI-Dungeon and Artbreeder are intentionally handicapped in ways that are designed to minimize the amount that users actually need to use the Deep Learning that makes them interesting.
Now, if we look at the Metaculus prediction for this question, the bulk of the probability mass is >=100 petaflops, which is ~10,000x more than GPT-3.
So, how much would we be willing to pay for access to such an AGI?
To make this more concrete, imagine that the first AGI is approximately as smart as the smartest human who ever lived. An obvious lower bound is "how much do really smart people make on average?". While this number varies widely from profession to profession, I think the fact that a Senior Software Engineer at Google makes somewhere around $250k/year $125/working hour is probably a decent estimate.
On the other hand, the upper-bound is probably something like "how much money do we have?". After all, Von-Neumann was responsible for ground-breaking innovations in fields such as quantum theory, the development of nuclear weapons, and the invention of the digital computer. Having access to the world's smartest person might literally be a matter of national survival.
If you consider that the Manhatten Project cost about 1% of the GDP of the US, that equals $227Billion/year or about $25million/hour.
Interestingly, if AGI really requires 100 petaflops, this number is not too far from the actual cost of running such an AGI. Computing on a GPU is estimated to cost between $0.03 and $0.30, which is $3million-$30million/hour for our hypothetical AGI (I have no idea why the range is so wide).
This suggests that we might be nearing the moment when a Manhattan project to build an AGI is reasonable, but we are nowhere near the point where commercial applications of AGI are feasible*.
*throughout this post, I assume that the development of AGI is primarily hardware-bound and that development of the first AGI will not lead to a hard takeoff. If the development of the first AGI does lead to recursive-self improvement followed shortly thereafter by a singularity, then the expected value of the first AGI is either insanely high or insanely negative (probably the latter though).