Wittgenstein ideas of what can't be "embedded" in language are indeed challenging our vision of what gradient based LLMs are capable of. Language is alive, changing, adapting, and language is an agent itself, even as tool. The ability to read not written context, or things that are of common knowledge, like how do you feel the air in your nose while arguing infuriated with a loved one, or the pain in the feet while standing in a queue, the sort of empathetic things that trigger actions in bodied beings. Language is mostly pragmatic, in origin and daily.
Wittgenstein ideas of what can't be "embedded" in language are indeed challenging our vision of what gradient based LLMs are capable of. Language is alive, changing, adapting, and language is an agent itself, even as tool. The ability to read not written context, or things that are of common knowledge, like how do you feel the air in your nose while arguing infuriated with a loved one, or the pain in the feet while standing in a queue, the sort of empathetic things that trigger actions in bodied beings. Language is mostly pragmatic, in origin and daily.