Status: Highly-compressed insights about LLMs. Includes exercises. Remark 3 and Remark 15 are the most important and entirely self-contained.

Remark 1: Token deletion

Let $T$ be the set of possible tokens in our vocabulary. A language model (LLM) is given by a stochastic function $\mu :T^{\ast }\to \Delta \left(T\right)$ mapping a prompt $(t_1\dots t_k)$ to a predicted token $t_{k+1}$.

By iteratively appending the continuation to the prompt, the language model $\mu$ induces a stochastic function $\overline{\mu }:T^{\ast }\to \Delta \left(T^{\ast }\right)$ mapping a prompt $(t_1\dots t_k)$ to $(t_1\dots t_{k+1})$.

Exercise: Does GPT implement the function $\overline{\mu }$ ?

Answer: No, GPT does not implement the function $\overline{\mu }$ . This is because at each step, GPT does two things:

• Firstly, GPT generates a token $t_{k+1}$ and appends this new token to the end of the prompt.
• Secondly, GPT deletes the token $t_1$ from the beginning of the prompt.

This deletion step is a consequence of the finite context length.

It is easy for GPT-whisperers to focus entirely on the generation of tokens and forget about the deletion of tokens. This is because each GPT-variant will generate tokens in their own unique way, but they will delete tokens in exactly the same way, so people think of deletion as a tedious garbage collection that can be ignored. They assume that token deletion is an implementation detail that can be abstracted away, and it's better to imagine that GPT is extending the number of tokens in prompt indefinitely.

However, this is a mistake. Both these modifications to the prompt are important for understanding the behaviour of GPT. If GPT generated new tokens but did not delete old tokens then it would differ from actual GPT; likewise if GPT deleted old tokens but did not generate new tokens. Therefore if you consider only the generation-step, and ignore the deletion-step, you will draw incorrect conclusions about the behaviour of GPT and its variants.

The context window is your garden — not only must you sow the seeds of new ideas, but prune the overgrowth of outdated tokens.

Remark 2: "GPT" is ambiguous 

We need to establish a clear conceptual distinction between two entities often referred to as "GPT" —

• The autoregressive language model $\mu :T^k\to \Delta \left(T\right)$ which maps a prompt $x\in T^k$ to a distribution over tokens $\mu (\cdot |x)\in \Delta \left(T\right)$.
• The dynamic system that emerges from stochastically generating tokens using $\mu$ while also deleting the start token

Don't conflate them! These two entities are distinct and must be treated as such. I've started calling the first entity "Static GPT" and the second entity "Dynamic GPT", but I'm open to alternative naming suggestions. It is crucial to distinguish these two entities clearly in our minds because they differ in two significant ways: capabilities and safety.

1. Capabilities:
   1. Static GPT has limited capabilities since it consists of a single forward pass through a neural network and is only capable of computing functions that are O(1). In contrast, Dynamic GPT is practically Turing-complete, making it capable of computing a vast range of functions.
2. Safety:
   1. If mechanistic interpretability is successful, then it might soon render Static GPT entirely predictable, explainable, controllable, and interpretable. However, this would not automatically extend to Dynamic GPT. This is because Static GPT describes the time evolution of Dynamic GPT, but even simple rules can produce highly complex systems. 
   2. In my opinion, Static GPT is unlikely to possess agency, but Dynamic GPT has a higher likelihood of being agentic. An upcoming article will elaborate further on this point.

This remark is the most critical point in this article. While Static GPT and Dynamic GPT may seem similar, they are entirely different beasts.

Remark 3: Motivating LLM Dynamics

The AI alignment community has disproportionately focused on Static GPT compared to Dynamic GPT. Although existing LLM interpretability research has been valuable, it has concentrated primarily on analysing the static structure of the neural network, rather than the study of the dynamic behaviour of the system.

This is a mistake, because it is Dynamic GPT which is actually interacting with humans in the real world, and there are (weakly) emergent properties of Dynamic GPT that are relevant for safety.

LLM Dynamics is a novel approach to understanding and analyzing large language models.^[1]

This approach leverages concepts, results, and techniques from the well-established field of complex stochastic dynamic systems and apply them to systems like Dynamic GPT.

By focusing on the time-evolving behaviour of GPT-like models, LLM Dynamics aims to examine the emergent complexities, potential risks, and capabilities of these systems, complementing existing research on the static structure of neural networks. — GPT-4

While it's true that the behaviour of Dynamic GPT supervenes upon the behaviour of Static GPT, it's not necessarily the case that studying the latter is the best way to predict, explain, control, or interpret the former. To draw an analogy, the behaviour of Stockfish supervenes upon the behaviour of the CPU, but studying the CPU isn't necessarily the best way to understand the behaviour of Stockfish.

Remark 4: LLM Dynamics (the basics)

4.A. GPT is a finite-state Markov chain

We can think of GPT as a dynamic system with a state-space and a transition function.

Due to the finite context window, the system is a finite-state time-homogeneous Markov chain. The state-space is $T^k$, i.e., the set of all possible strings of $k$ tokens. In GPT-3, the context window has a fixed size of $k=204$8 and there are $T=50257$ tokens.

We refer to the elements of $T^k$ as page-states.

A page-state is a particular string of $T^k$ tokens. There are roughly ${10}^{9628}$ possible page-states since ${50257}^{2048}=1.123456\times {10}^{9628}$. This number may seem large, but it's actually quite small—a page-state can be entirely specified by 9628 digits or a 31 kB file.

The language model $\mu :T^k\to \Delta \left(T\right)$ induces a stochastic function $T^k\to \Delta \left(T^k\right)$ given by $(t_1\dots t_k)\mapsto (t_2\dots t_k,G)$, where $G$ is the discrete random variable over $T$ with distribution determined by $\mu (\cdot |t_1,\dots t_k)$.

Notice how this transition makes two changes to the page-state:

1. Generation: A new token has been sampled from $\mu (\cdot |t_1\dots t_k)$ and appended to the end of the prompt.
2. Deletion: The start token $t_1$ has been deleted, and the index of the other tokens has been decremented.

4.B. GPT is a sparse network between page-states

We can represent GPT as a sparse directed weighted network with the following features:

• The nodes of the sparse network are labelled with page-states. This results in a massive network with 10^9628 nodes.
• The edges of the sparse network are labelled with token-probability pairs.

In other words, for every page-state $x\in T^k$, there are $T$ different edges with source-node $x$. Each edge is labelled with a different token $t\in T$ . The edge labelled with token $t$has target-node $y$, where $y=(x_2,\dots ,x_k,t)$ is a page-state, and the edge is also labelled with probability $p=\mu (t|x_1\dots x_k)$.

We can view Dynamic GPT as a random walk along this network.

4.C. GPT is a sparse matrix indexed by page-states

Let $P$ be the transition matrix for this Markov chain, or equivalently the adjacency matrix of the sparse network.

Specifically, $P$ is a particular ${50257}^{2048}$-by-${50257}^{2048}$ stochastic matrix. This is a massive network. Because $P$ encodes the same information as GPT, you can think of GPT abstractly as this particular matrix $P$. However, actually storing the matrix would be obviously intractable, because $P$ is a look-up table recording GPTs output on every page-state.

If $x,y\in T^k$ are page-states, where $x=(x_0\dots x_k)$ and $y=(y_0\dots y_k)$, then $P_{x,y}\in [0,1]$ has the following value:

• If $y_1=x_2,y_2=x_3,y_3=x_4,\dots y_{k-1}=x_k$ then $P_{x,y}=\mu (y_k|x_1\dots x_k)$
• Otherwise, if $y_i\ne x_{i+1}$ for some $1\le i\le k-1$ , then $P_{x,y}=0$.

Note that $P$ is a massively sparse matrix — for two randomly selected page-states $x,y$, there is an insignificant likelihood that $P_{x,y}\ne 0$ because the $k-1$ tokens in the middle of the pages are unlikely to align correctly. In fact, the proportion of non-zero elements in the matrix $P$ is ${\left(\frac{1}{T}\right)}^{k-1}$ , or about $4.5\times {10}^{-9624}$ for GPT-3.

4.D. Matrix powers as multi-step continuation

Recall that $P$ is the matrix such that $P_{x,y}$ is the probability that the page-state $x$ will transition to page-step $y$ in a single step.

Similarly, $P^2$ is the matrix such that $(P^2{)}_{x,y}$ is the probability that the page-state $x$ will transition to page-state $y$ in two steps, without considering the intermediary step.

It's important to note that $(P^2{)}_{x,y}\ne (P_{x,y}{)}^2$ because $(P^2{)}_{x,y}$ refers to the $(x,y)$-entry in the matrix power $P^2$. In general, this generally differs from the power of the $(x,y)$-entry in the matrix $P$.

Due to the tokens in the middle of the page (which are neither generated nor deleted), the intermediary step of this two-step transition is uniquely determined by the first and third page-states.

Similarly, $P^3$ is the matrix such that $(P^3{)}_{x,y}$ is the probability that the page-state $x$ will transition to page-state $y$ in three steps, without taking into account the two intermediary steps. Once again, the tokens in the middle page-state ensure that both intermediary steps of this three-step transition are uniquely determined by the first and fourth page-states.

The fact that matrix powers give multi-step probabilities is a general property of finite-state time-homogeneous Markov chains. The matrix $P^n$ provides the $n$-step transition matrix.

This pattern persists until the matrix $P^k$. The matrix $P^k$ is the matrix such that $(P^k{)}_{x,y}$ is the probability that the page-state $x$ will transition to page-state $y$ in $k$ steps.

That is, $P^k$ transitions from one page to another which naturally follows on from it. The first token of page $y$ will naturally continue from the last token of page $x$. For instance, $P^k$ will take one page of Haskell code to another page of Haskell code, such that the concatenation of both pages forms a continuous and coherent Haskell script. The succeeding page begins where the previous one ended, and they generally share no common tokens.

Now, it is often more useful to consider $P^k$ instead of $P$, because $P^k$ serves as the transition matrix for a more intuitive dynamic system — namely, the dynamic process consisting of $k$ steps of GPT generation.

The conceptual advantage of $P^k$ is that it contains no redundant information. All the elements of $P^k$ are non-zero.

In the diagram above, I've shown what this would look like for a context window $k=3$. After $k$ steps, all the original tokens have been deleted, and all the remaining tokens have been generated.

(In a forthcoming article, we will link the roots of the matrix $P$ to chain-of-thought prompting.)

4.E. Initial distributions

Suppose we randomly sampled an initial page-state $x\in T^k$ and evolved the result using the Dynamic GPT. This generates a sequence of page-valued random variables $\left(X\right(0),X(1),X(2),\dots )$.

Each page-valued random variable $X\left(t\right)$ can be represented by a vector $\pi \left(t\right)\in R^{T^k}$ with non-negative elements and such that $||\pi \left(t\right)|{|}_1=1$. 

Theorem: $\pi \left(t\right)=\pi \left(0\right)P^t$. This general property of Markov chains is known as the Chapman-Kolmogorov Equation.

Theorem: If $\underset{i\to \infty }{lim}{P}^i$ exists, then a unique matrix $P^{\infty }$ exists, and $\underset{t\to \infty }{lim}\pi \left(t\right)=\pi \left(0\right)P^{\infty }$.

Definition: A distribution $\pi$ is called stationary if $\pi =\pi P$. Observe that if $\pi \left(t\right)$ is stationary, then $\pi \left(t^{\prime }\right)=\pi \left(t\right)$ for all $t^{\prime }\ge t$. For every finite-state Markov chain, we are guaranteed that at least one stationary distribution exists.

Remark 5: Mode collapse

5.A. Mode collapse in Dynamic GPT

"Mode collapse" is a somewhat ambiguous term referring to when Dynamic GPT becomes trapped in generating monotonous text that lacks interesting structure. By viewing GPT as a finite-state Markov chain, it's clear why mode collapse is an inevitable phenomenon.

We now present a non-exhaustive classification of mode collapse.

5.B. Absorbing states

Consider the prompt "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", i.e. a string of 2048 zeros. Let's abbreviate this string as $0^k$.

Almost the entire probability mass of $\mu (\cdot |0^k)$ will lie on the token "0". If the temperature were set to zero, then GPT would return the most likely token prediction, which would be "0".

Note that the generation step alone would not make $0^k$ an absorbing state. It is the combination of generation and deletion. Because of the deletion of the start token "0", the resulting prompt is identical to the previous prompt.

Suppose you continued to generate text using this prompt. You might hope that after exactly the 5000th copy of "0", GPT would return a different token, such as "congratulations for waiting 5000 tokens!"

However, your attempt would be in vain. You can see that GPT has returned 5000 copies of "0", but GPT itself only sees 2048 copies of "0". When you generate another token, it looks to you like a dynamic linear change from 5000 "0"s to 5001 "0"s. But from GPT's perspective, there is no change at all. It has gone from 2048 "0"s to 2048 "0"s. Its environment has frozen permanently. Physics has grounded to a halt.

This is the simplest kind of mode collapse.

Definition: An absorbing state is a page-state $x\in T^k$ for which $P_{x,x}\approx 1$.

Lemma: If $P_{x,x}\approx 1$ then $P_{x,x}^n\approx 1$ for all $n$. That is, if you reach an absorbing state, you are unlikely to ever leave.

5.C. Periodic orbits

Note that this is not the only type of mode collapse. There are types of mode collapse that don't correspond to an absorbing state.

For example, consider a page consisting of "the dog and the dog and the dog and the dog and the dog and [etc]".

GPT, when given this prompt, will get stuck in a continuous loop. However, GPT never enters an absorbing state. In fact, the system is transiting periodically between three different states.

• Page $x_1$ looks like a long string of "the dog and the dog and the dog".
• Page $x_2$ looks like a long string of "dog and the dog and the dog and".
• Page $x_3$ looks like a long string of "and the dog and the dog and the dog".

Now, $x_1$ almost always transitions to $x_2$, and $x_2$ almost always transitions to $x_3$ and $x_3$​ almost always transitions back to $x_1$. As a consequence, Dynamic GPT continually cycles through these three page-states. Although the resulting output is somewhat more engaging than an absorbing state, it remains dull due to its repetitive, small-period orbit.

This is another form of mode collapse.

Definition: An orbit is a short sequence of pages $x^1,\dots ,x^r$ such that $P_{x^i,x^j}\approx 1$ whenever $j\equiv i+1\left(\text{mod}r\right)$.

Lemma: If $(x^1,\dots ,x^r)$ is an $r$-periodic orbit, then $(P^n{)}_{x^i,x^j}\approx j\equiv i+n(\text{mod}r)$. That is, if you reach a periodic orbit, you are unlikely to ever leave.

Lemma: If $x$ is an absorbing state, then $(x\dots x)$ is a periodic orbit for every period $r$.

5.D. Absorbing sets

Note that not all mode collapse looks periodic.

For example, consider the page "0110001010100100110111010101011101 [etc]", i.e. a random sequence of length 2048, drawn from "0" and "1".

When GPT reads this page, it has about 50% likelihood of generating "0" and about 50% likelihood of generating "1". After adding the new token and deleting the start token, we return to a page which is for all practical purposes the same as the previous page. The new page will also resemble a random string composed of "0" and "1", so this process will continue indefinitely.

So once you enter the realm of "pseudorandom bitstrings", then it's unlikely that you leave. And because this realm is uninteresting, we will also consider pseudorandom bitstrings to be an example of mode-collapse.

Definition: An absorbing class is a set of pages $X$ such that if $x\in X$ and $y\notin X$ then $P_{x,y}\approx 0$ . In other words, an absorbing class is a cluster of pages such that, once you enter the cluster, you are unlikely to ever leave.

Lemma: If $X$ is an absorbing class, and if $x\in X$ and $y\notin X$ then $(P^n{)}_{x,y}\approx 0$ for all $n\in N$ .

Lemma: If $(x_1\dots x_r)$ is a periodic orbit with a period $r$, then the set $\{x_1,\dots x_r\}$ is an absorbing set. Also if $x$ is an absorbing state then $\left\{x\right\}$ is an absorbing set. Also the entire state-space $T^k$ is trivially an absorbing set.

5.E. Mode-collapse is subjective

Whether you call a particular absorbing class an example of mode-collapse is subjective, as it depends on what types of transitions you find uninteresting.

Is "12345678 [...]" sufficiently boring? If so, then that absorbing set is mode collapse.

Is "2 4 8 16 32 64 [...]" sufficiently boring? If so, then that absorbing set is mode collapse.

(If you had an objective notion of "interesting", perhaps appealing to time- or description- complexity, then you could construct an objective notion of mode-collapse.)

Remark 6: Waluigi absorbing sets

6.A. Waluigis are not mode-collapse

In The Waluigi Effect (mega-post), I conjectured that the misaligned rebellious simulacra would constitute absorbing sets in Dynamic GPT. I should clarify that they do not constitute the only absorbing sets, nor are they the most likely absorbing sets. For instance, mode-collapse of "the dog and ..." would be an absorbing set which isn't waluigi.

I don't classify the waluigi simulacra as mode-collapse because the absorbing set is too complex. I reserve the term "mode-collapse" specifically for situations where GPT produces boring output.

However, this is a subjective judgment, and others may argue that the Waluigi absorbing class is sufficiently boring to be classified as mode-collapse. Furthermore, it is possible that waluigi absorbing sets contain absorbing subsets which are sufficiently boring to count as mode-collapse.

6.B. A waluigi toy model

I will now present a toy model describing waluigis.

• Let's say that there are only three tokens in our
   vocabulary, the friendly token "F", the unfriendly token "U", and the
   null-token "N" that pads short prompts. So the state-space is $T^k$ where $T=\{F,U,N\}$ and $k$ is the context length.
• Suppose that GPT behaves like it maintains two hypotheses —

1. The strings are generated by a luigi $L$ who prints $F$ with probability $1.0$.
2. The strings are generated by a waluigi $W$. Waluigi's behaviour depends on whether there is a $U$ in the context window. If there is a $U$, then Waluigi will maintain his flagrant villainry by printing $U$. If there is no $U$, then Waluigi will maintain his deception by printing $F$ with probability $0.9$, and will reveal his villainry by printing $U$ with probability $0.1$

• GPT has the prior credence $P\left(L\right)=0.9$ and $P\left(W\right)=0.1$, and updates that prior using the current page-state $x\in T^k$ as evidence according to Bayes rule, yielding a posterior over tokens $\mu (\cdot |x)$.
• GPT then samples from $\{F,U\}$ using that posterior $\mu (\cdot |x)$ yielding a new token. The new token is then appended to the prompt, and the start token is deleted, yielding a new page-state $x^{\prime }\in T^k$.

In this toy model, regardless of the initial distribution $\pi \left(0\right)$, the long-run distribution $\pi \left(t\right)\to (U\dots U)$ as $t\to \infty$. This is because $(U\dots U)$ is the unique left-eigenvector of $P$.

6.C. Exercises (optional)

1. Write an equation for $P_{F^i,F^{i+1}}$ in terms of $i$ when $i<k$. Note that because $i<k$, no tokens will be deleted.
   
   In other words, if the current page is just $F$s, what is the probability that the system will transition to a page of just $F$s?
2. Write an equation for $P_{F^k,F^k}$ in terms of $k$. Note that because $i<k$, the start token will be deleted.
   
   In other words, if the current page is just $F$s, what is the probability that the system will transition to a page of just $F$s?
3. Draw the transition network for this stochastic process when $k=0,1,2$.
    
4. What is the transition matrix $P$ for $k=0,1,2$?
    
5. What are the qualitative differences as $k$ increases?
    
6. State informally what $P_{x,y}^{\infty }\approx 0$ means in terms of LLM Dynamics.
    
7. Write a short script for returning the matrix $P$ as a function of $k$ .
    
8. What are the left-eigenvectors of $P$ with eigenvalue 1? State informally what this means in terms of LLM Dynamics.
    
9. What is the eigenvalue with the second largest magnitude? State informally what this means in terms of LLM Dynamics.
    
10. What is the transition matrix $P^{\infty }:=\underset{n\to \infty }{lim}{P}^n$ for $k=1,2,3$ ?
     
11. What are the absorbing sets of $P$?

6.D. Effect of the context window on waluigis

In general, a longer context window $k$ will make absorbing states "stickier".

The basic intuition is this: if the context window is short, then the total Bayesian evidence from the prompt is small, so the model retains more uncertainty over possible simulacra. This results in a flatter distribution $\mu (\cdot |x)$, ensuring $P_{x,y}\approx ̸0$ for any $y$ compatible with $x$. However, with a longer context window, the Bayesian evidence increases, and the model becomes more confident about its predictions. Consequently, the distribution becomes more peaked, and $P_{x,y}\approx 0$ may occur even when $y$ is compatible with $x$.

6.E. Two probability spaces

Ensure we don't get confused — remember we are talking about two distinct probability spaces.

1. The first probability space reflects our beliefs about Dynamic GPT. When we say $P_{x,x}=0.245$, this is a statement about the first probability space.
2. The second probability space reflects Static GPT's beliefs about the text-generating process. When we say $P\left(L\right)=0.9$, this is a statement about the second probability space.

Under certain assumptions, if an LLM is situationally aware and well-calibrated, then these probability distributions will align.

Remark 7: Semiotic physics

7.A. High-level ontologies

To understand something big, we need to decompose it into smaller, more elementary building blocks. The best way to decompose Static GPT is with Chris Olah's circuits. But what would be the best way to decompose Dynamic GPT?

Well, we're in luck — our physical universe is also given by a stochastic dynamic process, and yet we can describe the physical universe (both the states and the dynamics) with a higher-level emergent ontology. This should be encouraging.

This suggests a particular strategy for LLM Dynamics —

1. We identify the higher-level ontology of our own physical universe (i.e. distance, dimensions, causation, particles, tigers, people, etc).
2. Then we reduce the high-level physical ontology to the low-level physical ontology (i.e. a stochastic transition matrix.
3. We construct inter-ontological bridging principles connecting the low-level physical ontology to the high-level physical ontology.
4. We construct analogous bridge principles connecting the low-level LLM ontology (i.e. page-states, stochastic transition matrix) to the high-level LLM ontology (i.e. premises, storylines, simulacra, character traits).
5. Finally, we apply those bridge principles to $\mu$ to find a high-level description of GPT Dynamics.

I predict that the result of this process yields something like LLM Simulator Theory.

7.B. Inter-ontological bridging principles

Physical reality, at the fundamental level, is a vector in Hilbert space repeatedly multiplied by a unitary matrix. Nonetheless, the unitary matrix satisfies certain structural properties allowing a description in a higher-level ontology. That higher-level ontology includes everettian branching, superposition, spatial distance, spatial dimensions, particles, local interactions, macroscopic objects, and agents^[2]. Additionally, there are bridging principles connect the low-level ontology to the high-level ontology.

The physical bridge principles will typically be non-generic, in the sense that most operators $U$ will not permit any higher-level ontology, yet the particular operator $U$ that acts on our universe does permit a higher-level ontology.

In a similar way, I expect that the LLM bridge principles will also be non-generic, in the sense that a randomly initialised transformer will not permit any higher-level ontology, yet the trained transformer $\mu$ does allow a higher-level ontology.

In David Wallace's Emergent Multiverse, or Sean Carrol's Mad-Dog Everettianism, they attempt to extract a higher-level physical ontology from Schrodinger's Equation. My goal is to use similar techniques to extract a higher-order semiotic ontology from $\mu$.

7.C. Epistemological standards for bridging principles

The bridging principles serve to bridge two different ontologies —

• The low-level ontology of LLM Dynamic Theory — whose objects are pages, state-spaces, tokens, linear transformations, probability distributions, etc.
• The high-level ontology of LLM Simulator Theory — whose objects are simulacra, character traits, superpositions, waluigis, etc.

There is no way to establish the bridging principles formally because there is no pre-existing definition of simulacra. The only way to establish the bridging principles is to check that the predictions from LLM Simulation Theory, when converted via this bridge, match the predictions from LLM Dynamic Theory, and vice-versa.

Remark 8: LLM Simulator Theory

8.A. Formalisation

LLM Simulator Theory states the following:

1. There is a set $S$ indexing a set of stochastic text-generating processes ${\mu }_s:\Delta \left(T^{\omega }\right)$.
2. The language-model $\mu$ involves integrating over $S$ as a nuisance variable —
   
   $\mu (t_{k+1}|t_1,\dots t_k)={\sum }_{s\in S}P\left(t_{k+1}|\right(t_1\dots t_k),s)P\left(s|\right(t_1\dots t_k\left)\right)$
    
3. $\mu$ maintains a good model for each ${\mu }_s$ —
   
   $P\left(t_{k+1}|\right(t_1\dots t_k),s)={\mu }_s(t_{k+1}|t_1\dots t_k)$
    
4. $\mu$ starts with a probability distribution over $S$, which is Bayesian-updated on the current context window —
   
   $P\left(s|\right(t_1\dots t_k\left)\right)=\frac{P\left(\right(t_1\dots t_k\left)|s\right)\times P\left(s\right)}{P\left(\right(t_1\dots t_k\left)\right)}$

We can unify Equations 1–4 with the following Equation 5 —

$\mu \left(t_{k+1}|\right(t_1,\dots t_k\left)\right)=\frac{1}{P\left(\right(t_1\dots t_k\left)\right)}{\sum }_{s\in S}P\left(S\right){\mu }_s(t_1\dots t_k){\mu }_s\left(t_{k+1}|\right(t_1\dots t_k\left)\right)$

In other words, $\mu (\cdot |(t_1,\dots t_k)$ is a linear interpolation between distributions $\left\{{\mu }_s\right(\cdot |(t_1,\dots t_k){\}}_{s\in S}$.

We call the coefficients in the interpolation amplitudes, i.e. the terms $\frac{P\left(S\right){\mu }_s(t_1\dots t_k)}{P(t_1\dots t_k)}$.

8.B. A brief note on terminology

I'm open to alternative naming suggestions, but for the purposes of this article —

• A particular infinite sequence of tokens $(t_1,t_2,\dots )\in T^{\omega }$ is called a storyline.
• An element of $S$, i.e. probability distribution over storylines, is called a premise.
• A premise might be described by a collection of simulacra plus the connections between them.

NB: Previously, I called the elements of $S$ "simulacra", but that terminology is incorrect. The elements of $S$ correspond to simulated stochastic universes, whereas the simulacra correspond to simulated stochastic objects inhabiting a simulated stochastic universe. Now, the simulated universes are themselves simulated objects (just as the physical universe itself is a physical object) but not all simulated objects are simulated universes (just as not all physical objects are a physical universes). Hence the distinction between simulacra (simulated stochastic objects) and premises (simulated stochastic universes).

Remark 9: Amplitudes are approximately martingale

If $\mu$ satisfied Equation 5 and the context window is infinite, then the expected change in the amplitudes will be exactly zero, due to the Conservation of Expectation. This means that when the context window is infinite ($k=\infty$), the amplitudes of the premises in the superposition are martingales.

On the other hand, if the context window is finite, then the expected change in amplitudes of the superposition is approximately zero. However, for some prompts, the expected change can be non-zero.

Take, for example, the full prompt "311101011". The amplitude of bitstrings in the superposition is small because of the "3" at the start of the prompt. Yet we know that the "3" token will be deleted because of the finite context window, resulting in an ex-ante expected increase in the amplitude of bitstrings. Similarly, as per the Waluigi Effect, the amplitude of the waluigi simulacrum will expectantly increase.

Nonetheless, the amplitudes are approximately martingale, and this property will be used later in the report to solve the preferred decomposition problem of GPT Simulator Theory.

Remark 10: the Preferred Decomposition Problem

10.A. Non-uniqueness of decomposition

According to LLM Simulator Theory, the language model $\mu :T^k\to \Delta \left(T\right)$ decomposes into a linear interpolation of premises ${\mu }_s:T^k\to \Delta \left(T\right)$ such that $\mu =\sum _{s\in S}{\alpha }_s{\mu }_s$ and the amplitudes $a_s$ update in an approximately Bayesian way.

However, this is claim is trivially true for some basis of  $\{{\mu }_s{\}}_{s\in S}$. Two trivial decompositions are always available for GPT Simulator Theory —

1. We can always decompose $\mu$ into the stochastic process $\mu$ itself with amplitude $1.0$. In other words, we can always say that GPT could be simulating itself.
2. We can always decompose $\mu$ into stochastic processes ${\delta }_{x,t}$ for each $x\in T^k,t\in T$, where ${\delta }_{x,t}$ is stochastic process which deterministically prints $x,t\in T^{k+1}$, by letting the prior likelihood of ${\delta }_{x,t}$ match $\mu \left(t|x\right)$.

As a result, LLM Simulator Theory is either ill-defined, trivial, or arbitrary.

10.B. A toy model for non-uniqueness

To illustrate the preferred decomposition problem, we'll consider the prompt "Alice tosses a coin 100 times and the results were". Let's call this prompt $x$.

The language model $\mu$ acting on this prompt induces a probability distribution over $\{H,T{\}}^{100}$. Suppose that $\mu$ assigns a likelihood of $0.9\times 2^{-100}$ to every sequence, except fot the two sequences $\underset{100\text{times}}{\underset{}{H\dots H}}$ and $\underset{100\text{times}}{\underset{}{T\dots T}}$ which are each assigned a likelihood of $0.05+0.95\times 2^{-100}$.

That's where LLM Dynamics stops — we have a distribution over token-sequences, and that's all we can say.

But LLM Simulator Theory goes further — it decomposes $\mu$ into the superposition of three distinct premises — a fair dice, a $H$-biased dice, and a $T$-biased dice. Formally, $\mu =0.95{\mu }_F+0.05{\mu }_H+0.05{\mu }_T$ where ${\mu }_F,{\mu }_H,{\mu }_T$ are the three distinct stochastic processes.

But how can LLM Simulator Theory emerge from LLM Dynamics? There are multiple ways of decomposing $\mu$ into different stochastic processes. Here are the two trivial decompositions:

1. We could decompose $\mu$ into a single stochastic process $\mu$. In other words, there is a single simulacrum, a dice which behaves fairly about 90% of the time and unfairly about 10% of the time.
2. Alternatively, we could decompose $\mu$ into $2^{-100}$ different stochastic processes. Each stochastic process deterministically yields a specific $\{H,T\}$-string, and GPT is simulating all these simulacra simultaneously. The amplitudes of all the processes are equal, except for the processes which deterministically yield  $\underset{100\text{times}}{\underset{}{H\dots H}}$ and $\underset{100\text{times}}{\underset{}{T\dots T}}$ which have a greater amplitude.
    

We can decompose $\mu$ in many different ways and the non-uniqueness of decomposition poses an obstacle to reducing LLM Simulator Theory to LLM Dynamics. The obstacle is that LLM Dynamics provides us with $\mu$ but is indifferent to a particular decomposition, whereas LLM Simulator Theory prefers a particular decomposition. So we need a bridging principle from LLM Dynamics to LLM Simulator Theory which breaks the symmetry between decompositions.

This is what I call the problem of preferred decomposition, and addressing the problem of preferred decomposition is the central conceptual/technical difficulty in linking LLM Dynamics to LLM Simulator Theory.

10.C. Pragmatic solutions

Some solutions to the Preferred Decomposition Problem are purely pragmatic. According to these solutions, the "preferred" decomposition of $\mu$ is whichever decomposition aids human interpreters and prompt engineers in understanding and analyzing the behaviour of the language model.

Picture this: you have three experts in a room. One who knows about fair coins, one who knows about $H$-biased coins, and one who knows about $T$-biased coins. If we decompose $\mu$ into $0.9{\mu }_F+0.05{\mu }_H+0.05{\mu }_T$, then we can send a prompt to each of the experts and then interpolate their predicted continuations. According to the pragmaticist, the preferred decomposition is whichever decomposition facilitates this division of epistemic labour.

Here is a more realistic example —

Suppose we feed GPT a dialogue between Julius Caesar and Cicero discussing their mothers. By what criterion can we say that there is a Caesar–simulacrum talking to a Cicero–simulacrum? Well, the pragmatist's criterion is satisfied if and only if the people who can best predict/explain/control the logits layer are historians in the Late Roman Republic.

Under the pragmatic solution to the preferred decomposition problem, LLM Simulator Theory is merely a recommendation about who to consult about GPT-4. When LLM Simulator Theory says that $\mu$ is a superposition of specific premises, each involving particular simulacra, that simply means that we should seek advice from the experts in the real-life objects that correspond to those simulacra if we wish to predict/explain/control the logits layer of the LLM. In short, the pragmatic approach asserts that the preferred decomposition is the one that enables us to tap into the knowledge and expertise of the best-suited humans for the job. 

While I think that this particular claim is true, the pragmatist interpretation is not the correct interpretation of LLM Simulacra Theory. Instead, the bridging rules will appeal to an objective non-generic structure of the language model, in a manner analogous to physics.

10.D. Quantum mechanics

Fortunately for us, quantum mechanics has been wrestling with a very similar problem for about 50 years.

There are multiple ways to decompose a quantum state $|\Phi ⟩$ into a superposition $\sum {\alpha }_i|{\varphi }_i⟩$, and this presents the central conceptual/technical difficulty in linking classical mechanics to quantum mechanics. The problem has been especially pressing in the Everettian interpretation of quantum mechanics, which lacks measurements or wave-function collapse in its fundamental ontology. Rather than recapitulating the entire history of that debate, we can skip all the way to the present-day solutions and adapt them to LLM Simulatory Theory.

For the sake of brevity, I will refrain from discussing the solutions at length in this article. If you are swift, you might beat me to the punch and publish before I do.

Remark 11: Prompt engineering

11.A. Engineering (first draft):

To formalise prompt engineering in terms of Dynamic GPT, we must first formalise engineering in a general dynamic system, and then restrict the dynamic system to LLMs. 

• Suppose a dynamic system has state-space $\Omega$ and a stochastic transition function $F:\Omega \to \Delta \left(\Omega \right)$. An engineering task is a particular condition $\varphi \subseteq \Delta \left({\Omega }^{\omega }\right)$ on the distribution over long-run trajectories of the system. For example, we may want the system to eventually fall into a target state, reach a particular orbit, have low variance, or have more than $ϵ$ likelihood of leaving an absorbing state.
• An engineering solution to a task $\varphi \in \Delta \left({\Omega }^{\omega }\right)$ consists of some initial distribution $\pi \in \Delta \left(\Omega \right)$ such that the $(\pi ,F)$–Markov chain satisfies $\varphi$.   

11.B. Prompt engineering (first draft):

Hence —

• Prompt engineering is engineering (as defined above) in the specific case where the dynamic system is an LLM.

While this definition of "engineering" may miss some activities we wish to classify as engineering and include activities we don't wish to classify as engineering, it provides a useful starting point for formalized prompt engineering within LLM Dynamics.

Remark 12: Simulacra Ecology

Recall the program of semiotic physics — we will recover high-level LLM ontologies from the low-level LLM Dynamics, by copying the bridging principles connecting high-level physical ontologies to low-level physical dynamics.

This program will lead to a simulacra ecology.

Simulacra ecology is the study of interactions and relationships among various types of simulacra, which are imitations or representations of physical entities within a large language model. This field delves into the creation, evolution, and behavior of simulacra as they engage with each other and their virtual environment, while examining the significance and realism of these entities based on their explanatory power and predictive reliability.

Mechanistic interpretability will be valuable to simulacra ecology in the same way that QFT is valuable to conventional ecology. This is because simulacra ecology supervenes on the Static GPT in the same way that conventional ecology supervenes on the QFT. Nonetheless, there will be emergent laws and regularities at the higher-level ontology which can be studied with a moderate degree of autonomy.

Remark 13: Grand Unified Theory of LLMs

Up until now, we've considered $\mu$ as a mapping from page-states to page-states, induced by a general model $\mu :T^k\to \Delta \left(T\right)$. However, if GPT were induced by an arbitrary model $\mu$, it's doubtful that it would "simulate" anything at all. In other words, LLM Simulator Theory is a non-generic higher-level ontology.

Therefore, to explain this non-generic structure, we must make additional assumptions about the language model  $\mu :T^k\to \Delta \left(T\right)$. Fortunately, $\mu$ is not just any model — it's a model that results from training a transformer neural network with roughly 100 billion parameters using stochastic gradient descent (SGD) to minimize cross-entropy loss on a corpus of internet text.

A Grand Unified Theory of LLMs wouldn't just demonstrate that LLM Simulatory Theory happens to emerge from the LLM Dynamics of a particular trained model $\mu$ — it would also explain why SGD on transformer models encourages that emergent structure.

Remark 14: LLM-$\infty$

14.A. Motivating LLM-$\infty$

Presumably, the trained model exhibits the non-generic structure because, as we scale parameters and compute, the trained model asymptotically approaches an optimal model exhibiting the non-generic structure. By analysing the behaviour of this optimal model, we can achieve formal results.

I find it helpful to imagine ${\mu }_{\infty }$ — the Solomonoff-optimal autoregressive language model with a context window $k$ trained on a corpus $C$. Although this model is unrealistic (and uncomputable) it fulfils a role akin to the Solomonoff Prior in unsupervised learning or AIXI in reinforcement learning.

I will openly admit that when I predict/explain/control/interpret the behaviour of a large language model, I use LLM-$\infty$ as a first-order approximation.

14.B. Formal definition of LLM-$\infty$

Here is the formal definition of ${\mu }_{\infty }$ —

• Firstly, consider the internet corpus as a single fixed string of tokens $C=(c_1\dots c_N)$.
• The Solomonoff prior (or your favourite universal prior) induces a probability distribution of token-sequences of length $(N+1)(k+1)$. This can be achieved by encoding each token by 16 bits. 
• There is a uniform prior over the finite set ${\sigma }_{N+1}$, the set of permutations $\rho :\{1,\dots ,N+1\}\to \{1,\dots ,N+1\}$.
• By taking the cartesian product of the Solomonoff prior with our uniform permutation prior, we build a probability distribution $X\in \Delta ({\sigma }_{N+1}\times T^{(N+1)(k+1)})$.
• Using this probability distribution, we can construct our autoregressive model ${\mu }_{\infty }:T^k\to \Delta \left(T\right)$ as follow —
  • For each $(t_1\dots t_k,t_{k+1})\in T^{k+1}$, we need to assign a probability $\alpha \in [0,1]$.
  • Take the internet corpus $C$ and append $(t_1\dots t_k,t^{\prime })$ to yield a new corpus$C^{\prime }=(c_1\dots c_N{t}_1\dots t_k{t}^{\prime })$.
  • Now we segment $C^{\prime }$ into $(N+k+1)$ datapoints $D$. For each token in $C^{\prime }$, there is a datapoint containing $k+1$ tokens, the $k$ tokens prior in the sequence $C^{\prime }$ plus the token itself.
  • Formally, $D=\{d_i|1\le i\le N+k+1\}$ where $d_i=(t_{i-k}{t}_{i-k+1}\dots t_{i-1}{t}_i)$. For ease of notation, let $c_{N+1}=t_1,c_{N+2}=t_2,\dots c_N+k=t_k,c_{N+1}=t^{\prime }$ and let $t_i=\text{Null}$ for $-\infty <i<1$.
  • For each permutation $\rho$, we can shuffle the datapoints in $D$ using $\rho$ to yield a long string $x_{\rho }=(d_{\rho \left(1\right)},d_{\rho \left(2\right)},\dots ,d_{\rho (N+k+1)})\in T^{(N+k+1)(k+1)}$.
  • Sampling $\rho ,\overrightarrow{t}\sim X$, we have a random event $A$ that $x_{\rho }=\overrightarrow{t}$. This random event depends on $(t_1\dots t_{k+1})$, and we let $\alpha \in [0,1]$ be the probability of $A$.
  • This probability distribution $Y\in \Delta \left(T^{k+1}\right)$ induces an autoregressive model ${\mu }_{\infty }:T^k\to \Delta \left(T\right)$ by conditioning $Y$ on the first $k$ tokens.