The big question about GPT-3 is not whether it can reproduce everything it's ever read, but whether it gleaned some understanding of the world while reading. Answering the question about the Channel with something like "No one ever walked across the Channel since humans can't walk on water" would indicate that it did, even though it'd be factually wrong.
The big question about GPT-3 is not whether it can reproduce everything it's ever read, but whether it gleaned some understanding of the world while reading. Answering the question about the Channel with something like "No one ever walked across the Channel since humans can't walk on water" would indicate that it did, even though it'd be factually wrong.