I'm figure out if what I think of "thinking" is fundamentally different from what you describe here - you make me think that these LLMs have a more impressionistic view of the world, where "5 + 3" gets the right answer if you've related that expression to "8" tightly enough, or you have a tight enough definition of "fiveness" "threeness" and "plus-age" you'll arrive at "8". When I do 5 + 3 or a geometric proof or analyze an if-then statement, I feel like I'm doing more than just this impressionistic closeness ranking. Do you think I'm fooling myself, or is...
Next time someone clearly needs a nap I'm going to ask them if they've tried turning themselves off and then on again