I really don't care about IQ tests; ChatGPT does not perform at a human level. I've spent hours with it. Sometimes it does come off like a human with an IQ of about 83, all concentrated in verbal skills. Sometimes it sounds like a human with a much higher IQ than that (and a bunch of naive prejudices). But if you take it out of its comfort zone and try to get it to think, it sounds more like a human with profound brain damage. You can take it step by step through a chain of simple inferences, and still have it give an obviously wrong, pattern-matched answer at the end. I wish I'd saved what it told me about cooking and neutrons. Let's just say it became clear that it did was not using an actual model of the physical world to generate its answers.

Other examples are cherry picked. Having prompted DALL-E and Stable Diffusion quite a bit, I'm pretty convinced those drawings are heavily cherry picked; normally you get a few that match your prompt, plus a bunch of stuff that doesn't really meet the specs, not to mention a bit of eldritch horror. That doesn't happen if you ask a human to draw something, not even if it's a small child. And you don't have to iterate on the prompt so much with a human, either.

Competitive coding is a cherry-picked problem, as easy as a coding challenge gets... the tasks are tightly bounded, described in terms that almost amount to code themselves, and come with comprehensive test cases. On the other hand, "coding assistants" are out there annoying people by throwing really dumb bugs into their output (which is just close enough to right that you might miss those bugs on a quick glance and really get yourself into trouble).

Self-driving cars bog down under any really unusual driving conditions in ways that humans do not... which is why they're being run in ultra-heavily-mapped urban cores with human help nearby, and even then mostly for publicity.

The protein thing is getting along toward generating enzymes, but I don't think it's really there yet. The Diplomacy bot is indeed scary, but it still operates in a very limited domain.

... and none of them have the agency to decide why they should generate this or that, or to systematically generate things in pursuit of any actual goal in a novel or nearly unrestricted domain, or to adapt flexibly to the unexpected. That's what intelligence is really about.

I'm not saying when somebody will patch together an AI with a human-like level of "general" performance. Maybe it will be soon. Again, the game-playing stuff is especially concerning. And there's a disturbingly large amount of hardware available. Maybe we'll see true AGI even in 2023 (although I still doubt it a lot).

But it did not happen in 2022, not even approximately, not even "in a sense". Those things don't have human-like performance in domains even as wide as "drawing" or "computer programming" or "driving". They have flashes of human-level, or superhuman, performance, in parts of those domains... along with frequent abject failures.

"Slow takeoff" means things accelerate and go crazy before we get to human-level AGI. It does not mean that after we get to human-level AGI, we still have some non-negligible period where they are gradually getting smarter and available for humans to study and interact with. (See discussion)

Quoting Paul from that conversation, emphasis mine:

My intuition is that by the time that you have an AI which is superhuman at every task (e.g. for $10/h of hardware it strictly dominates hiring a remote human for any task) then you are likely weeks rather than months from the singularity.

But mostly this is because I think "strictly dominates" is a very hard standard which we will only meet long after AI systems are driving the large majority of technical progress in computer software, computer hardware, robotics, etc. (Also note that we can fail to meet that standard by computing costs rising based on demand for AI.)

My views on this topic are particularly poorly-developed because I think that the relevant action (both technological transformation and catastrophic risk) mostly happens before this point, so I usually don't think this far ahead.

As I understand it, Paul Christiano, Ajeya Cotra, and other timelines experts who continue to have 15+ year timelines continue to think that we are 15+ years away from the existence of software/AIs which can run on hardware for $10/hr and are more productive/useful than hiring a remote human worker for any (relevant) task.

As much as I agree that things are about to get really weird, that first diagram is a bit too optimistic. There is a limit to how much data humanity has available to train AI (here), and it seems doubtful we can make use of data x1000 times more effectively in such a short span of time. For all we know, there could be yet another AI winter coming - I don't think we will get that lucky, though.

There's a huge difference between "we have AI methods that can solve general problems" and "we have AI methods that, when wielded by a team of the world's leading AI engineers for months or years of development, can solve general problems". Most straightforwardly, there is a very limited supply of the world's leading engineers and so this doesn't scale.

One possibility is that AI capabilities might proceed to become easier to wield over time, so that more and more people can wield it, and it takes less and less time to achieve useful things using it.

Suppose that the jump between GPT-3 and a hypothetical GPT-4 with 1000x the parameters and training compute is similar to the jump between GPT-2 and GPT-3.

This assumption seems absurd to me; per Chinchilla scaling laws, optimal training of a (dense) 100T parameter model will cost > 2.2Mx Gopher's cost and require > 2 quadrillion tokens.

We won't be seeing optimally trained hundreds of trillions dense parameter models anytime soon.

If we place "average human intelligence" at the level of GPT-3 (or the similar sized open source BLOOM model), then such an AGI can currently be bought for $120k.

Likewise, I don't understand this assumption either. GPT-3 is superhuman in some aspects and subhuman in others; I don't think it "averages out" to median human level in general.

Human level is not AGI. AGI is autonomous progress, potential to invent technology from distant future if given enough time.

The distinction is important for forecasting the singularity. If not developed further, modern AI never gets better on its own, while AGI eventually would, even if it's not capable of direct self-improvement initially and needs a relatively long time to get there.

The important thing to notice is that all existing AIs are completely devoid of agency. And this is very good! Even if continued development of LLMs and image networks surpasses human performance pretty quickly, the current models are fundamentally incapable of doing anything dangerous of their own accord. They might be dangerous, but they're dangerous the way a nuclear reactor is dangerous: a bad operator could cause damage, but it's not going to go rogue on its own.

Rumors are GPT-4 will have less than 1T parameters (and possibly no larger than GPT-3) - unless Chinchilla turns out to be wrong or obsoleted apparently this is to be expected. 

"Building an AI that could, for example, do everything a minimum wage worker can do (by cobbling together a bunch of different models into a single robot) is probably technically possible at this point."

Completely disagree with this. I don't think we're anywhere near AI having this kind of versatility. Everything that is mentioned in this article is basically a more complex version of Chess. AI has been better than humans at chess for over 20 years and so what? Once AI is outside of the limited domain on which it was trained, it is completely lost. There is absolutely no evidence that 'cobbling together a bunch of different models into a single robot' will fix the problem.

Now you can of course call a self-checkout machine 'doing everything a minimum wage worker can do', so to sidestep this, let's say we specifically want a secretary. Someone who should be able to reliably accomplish basic, but possibly non-standard tasks. I would not bet on robot secretaries coming to market in the next 20 years.

Ask ChatGPT to prove that the square root of 2 is not rational. It starts out reasonably but soon claims that 2 is not a rational number! It's justification for that is that the square root of two is not rational.

Even though the "G" in AGI stands for "general", and even if the big labs could train a model to do any task about as well (or better) than a human, how many of those tasks could be human-level learned by any model in only a few shots, or in zero shots? I will go out on a limb and guess the answer is none. I think this post has lowered the bar for AGI, because my understanding is that the expectation is that AGI will be capable of few- or zero-shot learning in general.