Watching this video, (Edit: around 25:50) I was pleased to see a specific prediction made by the person claiming that GPT-3 couldn't do reasoning: He claimed that GPT-3's ability to unscramble a word was not reasoning but mere pattern-matching and that he would be very surprised if GPT-3 could scramble an arbitrary word. He said that if it could do that, that would be evidence that it really was reasoning.
Anyone know of other prominent (controversial, important) public predictions of what GPT-3 can or can't do? What about GPT-4? Or GPT-2? It would be interesting to compile a list and see if any patterns jump out, e.g. people consistently over or under-estimating it.