Cross-posted from New Savanna.

The method of loci is a memory technique used from ancient Greece and Rome up through the Middle Ages, into the Early Modern Era, and still in use today, though sporadically. Rather than explain it myself, or force you to read the Wikipedia entry, though you’re welcome to do so, I thought I’d like my good buddy, Schoolmaster Chat explain it to you. After that I’ll made a few observations about why it was necessary in the first place and conclude with a remark about transformers.

The Method of Loci

What can you tell me about the classical memory technique known as the method of loci?

The method of loci, also known as the memory palace technique, is a classical mnemonic device dating back to ancient Greece and Rome. It is a memory enhancement strategy that involves associating the information you want to remember with specific locations in a familiar space, such as a building or a room. The term "loci" is Latin for "places."

Here's a basic overview of how the method of loci works:

• 1. Choose a Familiar Setting: Select a place you are very familiar with, such as your house, a route you frequently walk, or a room you know well. This place will serve as your "memory palace."
   
• 2. Mentally Navigate the Space: Take a mental walk through this space and identify specific locations or landmarks. These could be furniture, doors, windows, or any distinct features within the environment.
   
• 3. Associate Information with Locations: Associate the information you want to remember with specific locations in your chosen space. Create vivid and memorable mental images that link the information to each location.
   
• 4. Mentally Walk Through the Space: When you need to recall the information, mentally walk through your memory palace. As you reach each location, the associated information should come to mind.

The effectiveness of the method of loci is based on the human brain's natural ability to remember spatial relationships and locations. By tying information to specific places, you create a mental map that makes it easier to retrieve the information later.

This technique has been used by ancient orators, scholars, and individuals with impressive memory feats. It's a powerful tool for memorizing lists, speeches, or any sequential information. While it may take some practice to master, many people find it to be a valuable and creative approach to memory enhancement.

Remarks, transformers, evaluating AIs

The obvious question – Why go to all that trouble? – has an obvious answer – Because remembering long collections of things is difficult. Why is it difficult? For my present purposes it’s sufficient to say: Because that’s just not how the brain works. Why that’s so, that’s a different question. Psychologists have spent a great deal of time investigating human memory, but we need not go into that.

I will not in passing, however, that we also find arithmetic calculation difficult. Oh, the concepts are trivial. That’s not where the difficulty lies. It’s in the practical application of those concepts in the act of calculation. Children are drilled for hours and hours over years and years in doing such calculation – though I would hope less so now that cheap calculations (not to mention computers) are readily available to do that work. Learn to do it, yes. It's an important basic skill, upon which important concepts can be piggy-backed. But endless drill, no longer necessary.

Which brings us to transformers. Getting them to be fluent with multidigit arithmetic has proven to be an interesting and challenging problem. It’s not that anyone wants them to do it in practical applications, rather, to put it bluntly, investigators are trying to see how human their capacities are. Well, the fact that multidigit arithmetic is difficult for transformers seems to make the very human, doesn’t it? After all, numeric calculation doesn’t even exist in pre-literate cultures. It’s not part of our biological endowment the way language is.

Now, to transformers. My first point is simple: How does the method of loci work? Once you’ve established your memory palace, and then stocked it with visual associations to your information, you execute this “program,” if you will, but starting at the beginning and walking through to the end. There’s no looping, no branching, just a single stream of recall. And that’s like the single stream of token generation that transformers produce.

My second point is that you have to set your memory palace up before you can use it. I supposed that’s Phase I of pretraining. Once that’s been done you can enter Phase II where you specialize the palace for a specific use by stocking it with images linked to the items you want to remember. I suppose it would be pushing things a bit to liken this to RLHF, but I’m not aiming for anything precise here. Just a crude correspondence.

One final and general observation. Pretty much everything I read about the capacities of LLMs vis-à-vis human capacities seems to assume that there is some one thing that is human mental capacity. No, there isn’t. There is a biological endowment, the nature of which is not clear. Over that, however, there is quite a bit of cultural modification and extension.

Well, guess what, just as arithmetic isn’t part of our native biological endowment, neither are detailed logical reasoning or (complex) planning. Those are cultural extensions and elaborations. That doesn’t mean that we an effective AI doesn’t have to capable to doing those things. Sure, for some applications they’re essential. But it’s foolish to routinely and unthinkingly include those capacities in the process of assessing whether or not an AI has reached “human capacity.” The whole field needs a much more sophisticated understanding of human behavior in order to begin making such judgments in a sensible way. Without such understanding The Spirit of St. Turing is flying blind.

More later.