Great post. I am not sure I'd go as far as “abstract understanding”, feels more like "Extreme memorization" to me. Yet there's no question that there is *some kind* of understanding.
As far as qualia, yes, never say never, but seems to me the word can only make sense and be used within a general AI framework where concepts like causality, recursion, self-reference, etc ... have been cracked and implemented. Something that GPT-2 (and the entire AI field) is far, far from having achieved.
Great post. I am not sure I'd go as far as “abstract understanding”, feels more like "Extreme memorization" to me. Yet there's no question that there is *some kind* of understanding.
As far as qualia, yes, never say never, but seems to me the word can only make sense and be used within a general AI framework where concepts like causality, recursion, self-reference, etc ... have been cracked and implemented. Something that GPT-2 (and the entire AI field) is far, far from having achieved.
Same points could also be made about the remarkable Aristo ( https://arxiv.org/abs/1909.01958 )