On the claim “We get the sense of words from how they relate to one another in sentences”.

It will be interesting to look into this theory as it is a theory of how computers get the meanings of words. We will see that the idea of “meaning” or “sense” of words here is impoverished.

To assess the theory, let’s consider where a child gets the sense of e.g. the word “dog” from. If the sense comes from sentences, then it apparently gets the sense from e.g. descriptions of dogs or general talk that includes the word “dog”. The child never has to see a dog in the real world. All the information comes through the sentences.

Consider some general talk involving the word: “that dog is running quickly”; “dogs have four legs, fur and a tail”; “my dog is brown”. We’re supposed to get the sense of the word by how it fits in with other words in sentences, like the words “is running quickly”, “has four legs, fur and a tail”, “is brown”. But in order for that to work, we must already know the sense of those other words “tail”, “fur”, “leg”, “brown”, “run”, etc. And how did we learn the sense of those words?

According to the theory we’re assessing here, we must have gotten the sense from the way each word fits in with other words in sentences. But, as with the “dog” example, getting the sense of a word depends on already knowing the sense of other words. In order to learn the sense of a word we must know the sense of other words. But we started (at birth) not knowing the sense of any word. As we start with no meanings of words at all, and the meaning of one word is obtained from the meanings of other words, there is no space in the theory for “meaning” to enter in in the first place. The theory thus fails.

Let’s explore the idea from another angle. Say a child points at a dog in the real world and asks its parent “dog?” and the parent responds “yes”. This or something analogous may be the manner in which people link words with meaning. Such an occurrence is not the acquisition of meaning purely from looking at how words relate to each other in sentences. The event does not occur within the boundaries of sentences and language. It involves looking and interacting with objects in the real world.

Let’s turn this example into an alternative version to consider how words might or might not have meanings for AI. There is never any step at which an AI system looks at a dog, and asks “dog?” and gets a response. (I’m talking about AI that handles language.)

Imagine the following bizarre scenario. Say that instead of dogs and other real-world objects, a child’s world is full exclusively of featureless things—for the sake of having an object to provide some mental imagery, let’s say everything around it is white boxes. The parent with the child points to a white box and says “dog” and then to a different white box and says “tree” and then at another and says “hat”. And let’s say, in some odd experiment with parents playing along, the parents have normal conversations about everyday things, pretending that the white boxes around them are various objects of regular life. So their sentences are sentences a child might hear in regular life: “Let’s go into that store and get some of that bread in the window”; “look at that dog chasing that ball” etc. But all the objects they draw the child’s attention to are white boxes. This way the child is exposed to ordinary language but nothing else.

If we get the sense of words through sentences, the child in such circumstances would come to understand the words “dog”, “bread”, “ball” etc. just as a child living in the real world might.

But I do not think they would. All they have is white boxes, not the variety of objects in the real world, which might convey meanings to the child through their shape, colour, behaviour, etc. And, as we’ve seen above, sentences alone cannot convey word meanings. There is no avenue through which meanings of words can come to the child.

This is what “meanings” are to AI. Every object or word to an AI system is a white, featureless box. Put differently, a word has no meaning for computers. They construct sentences by looking at tally counts of word frequences and choosing the words that have appeared most often in past text.

Counting up the number of times words have appeared near other words (which involves no interaction with real world objects or concepts) again involves no link between a word and a meaning, such as the link a child might form when it connects a word with a real object in the world by looking and pointing at the dog.

.

.

.

Take a look at my review of AI inventor Geoffrey Hinton's ideas on machine consciousness or other articles on the scope and limits of AI here.