Apparently, a human (Kellin Pelrine, a solid player but not even a Go professional) was able to beat some state-of-the-art Go AIs (KataGo and Leela Zero) by learning to play an adversarial policy found using RL. Notice that he studied the policy before the match and didn't receive any AI advice during play.
I'm not surprised adversarial policies for Go AIs are possible, this is in line with previous results about RL and adversarial examples more generally. I am surprised this adversarial policy is teachable to humans without colossal effort.
This is some evidence against the "scaling hypothesis", i.e. evidence that something non-trivial and important is missing from modern deep learning in order to reach AGI. The usual counterargument to the argument from adversarial examples is: maybe if we could directly access a human brain, we could find adversarial examples against humans. I can believe that it's possible to defeat a Go professional by some extremely weird strategy that causes them to have a seizure or something in that spirit. But, is there a way to do this that another human can learn to use fairly easily? This stretches credulity somewhat.
Notice also that (AFAIK) there's no known way to inoculate an AI against an adversarial policy without letting it play many times against it (after which a different adversarial policy can be found). Whereas even if there's some easy way to "trick" a Go professional, they probably wouldn't fall for it twice.
Just to make sure we’re on the same page, Fig. 4.1 was about training the model by gradient descent, not in-context learning. I’m generally somewhat averse to the term “in-context learning” in the first place; I’m skeptical that we should think of it as “learning” at all (as opposed to, say, “pointing to a certain task”). I wish people would reserve the term “learning” for the weight updates (when we’re talking about LLMs), at least in the absence of more careful justification than what I’ve seen.
In particular, instead of titling the paper “Language Models are Few-Shot Learners”, I wish they had titled it “Language Models Can Do Lots of Impressive Things Without Fine-Tuning”.
But Fig. 4.1 of that paper is definitely about actual learning.
In order for it to be a strong counterargument to this example, it should be the case that an LLM can learn to play Go on superhuman level …
I think there are so many disanalogies between LLMs-playing-Go and humans-playing-Go that it’s not even worth thinking about. ¯\_(ツ)_/¯ For example, humans can “visualize” things but LLMs can’t (probably). But OTOH, maybe future multi-modal next-gen LLMs will be able to.
More generally, I haven’t seen any simple comparison that provides air-tight evidence either way on sample-efficiency of deep learning versus human brains (and “deep learning” is itself is big tent—presumably some model types & sizes are more sample-efficient than others).
As it happens, I do believe that human brains are more sample efficient than any deep learning model. But my reasons for believing that are pretty indirect and I don’t want to talk about them.
See also article in Financial Times
Apparently, a human (Kellin Pelrine, a solid player but not even a Go professional) was able to beat some state-of-the-art Go AIs (KataGo and Leela Zero) by learning to play an adversarial policy found using RL. Notice that he studied the policy before the match and didn't receive any AI advice during play.
I'm not surprised adversarial policies for Go AIs are possible, this is in line with previous results about RL and adversarial examples more generally. I am surprised this adversarial policy is teachable to humans without colossal effort.
This is some evidence against the "scaling hypothesis", i.e. evidence that something non-trivial and important is missing from modern deep learning in order to reach AGI. The usual counterargument to the argument from adversarial examples is: maybe if we could directly access a human brain, we could find adversarial examples against humans. I can believe that it's possible to defeat a Go professional by some extremely weird strategy that causes them to have a seizure or something in that spirit. But, is there a way to do this that another human can learn to use fairly easily? This stretches credulity somewhat.
Notice also that (AFAIK) there's no known way to inoculate an AI against an adversarial policy without letting it play many times against it (after which a different adversarial policy can be found). Whereas even if there's some easy way to "trick" a Go professional, they probably wouldn't fall for it twice.