Status: Highly-compressed insights about LLMs. Includes exercises. Remark 3 and Remark 15 are the most important and entirely self-contained.

Remark 1: Token deletion

Let $T$ be the set of possible tokens in our vocabulary. A language model (LLM) is given by a stochastic function $μ : T^{*} \to Δ (T)$ mapping a prompt $(t_{1} \dots t_{k})$ to a predicted token $t_{k + 1}$ .

By iteratively appending the continuation to the prompt, the language model $μ$ induces a stochastic function $¯ μ : T^{*} \to Δ (T^{*})$ mapping a prompt $(t_{1} \dots t_{k})$ to $(t_{1} \dots t_{k + 1})$ .

Exercise: Does GPT implement the function $¯ μ$ ?

Answer: No, GPT does not implement the function $¯ μ$ . This is because at each step, GPT does two things:

Firstly, GPT generates a token $t_{k + 1}$ and appends this new token to the end of the prompt.
Secondly, GPT deletes the token $t_{1}$ from the beginning of the prompt.

This deletion step is a consequence of the finite context length.

It is easy for GPT-whisperers to focus entirely on the generation of tokens and forget about the deletion of tokens. This is because each GPT-variant will generate tokens in their own unique way, but they will delete tokens in exactly the same way, so people think of deletion as a tedious garbage collection that can be ignored. They assume that token deletion is an implementation detail that can be abstracted away, and it's better to imagine that GPT is extending the number of tokens in prompt indefinitely.

However, this is a mistake. Both these modifications to the prompt are important for understanding the behaviour of GPT. If GPT generated new tokens but did not delete old tokens then it would differ from actual GPT; likewise if GPT deleted old tokens but did not generate new tokens. Therefore if you consider only the generation-step, and ignore the deletion-step, you will draw incorrect conclusions about the behaviour of GPT and its variants.

The context window is your garden — not only must you sow the seeds of new ideas, but prune the overgrowth of outdated tokens.

Remark 2: "GPT" is ambiguous

We need to establish a clear conceptual distinction between two entities often referred to as "GPT" —

The autoregressive language model $μ : T^{k} \to Δ (T)$ which maps a prompt $x \in T^{k}$ to a distribution over tokens $μ (\cdot | x) \in Δ (T)$ .
The dynamic system that emerges from stochastically generating token

...

Simulators