I feel like the missing probability should come from indistinguishability, but not because of programs halting. Consider a countable family of possible discrete transitions on a potentially infinite state space. Maybe a programming language is a way to enumerate that space of transitions, but the language itself enumerates in a partially redundant manner. For instance f(x)=x+x versus g(x)=2*x. So you will never be able to distinguish between those programs by watching the transition sequence [1,2,4,8,...]. In some sense the PL story of alpha, beta, gamma ... equalities is a way to describe equivalence classes of extensionally equal terms, but for a Turing complete language you will never find a decision procedure to unify pairs of terms p and q iff forall x, p(x)=q(x). So in some sense this countable family of state space transitions, if enumerated by a programming language, must remain indistinguishable.

This repo enumerates and runs programs in the iota combinator language. I wrote it many years ago, maybe it still works

https://github.com/scottviteri/ExploringTheSpaceOfShortPrograms

I really like the idea of finding steering vectors that maximize downstream differences, and I have a few follow-up questions.

Have you tried/considered modifying c_fc (the MLP encoder layer) bias instead of c_proj (the MLP decoder layer) bias? I don't know about this context, but (i) c_fc makes more intuitive sense as a location to change for me, (ii) I have seen more success playing with it in the past than c_proj, and (iii) they are not-equivalent because of the non-linearity between them.

I like how you control for radius by projecting gradients onto the tangent space and projecting the steering vector of the sphere, but have you tried using cosine distance as the loss function so there is less incentive for R to naturally blow up? Let $D\left(z\right)={\sum }_{i=1}^n{\sum }_{t\in I_i}cosDist(Z_{{\ell }_{target},i,t}\left(z\right),Z_{{\ell }_{target},i,t}\left(0\right))$in ${max}_{z}D\left(z\right)$.

When you do iterative search for next steering vectors, I do not expect that constraining the search to an orthogonal subspace to previously found steering vectors to be very helpful, since the orthogonal vectors might very well be mapped into the same downstream part of latent space. Since the memory demands are quite cheap for learning steering vectors, I would be interested in seeing an objective which learned a matrix of steering vectors simultaneously, maximizing the sum of pairwise distances. Suppose we are learning $K$ vectors simultaneously.
${max}_{z_1,\dots ,z_K}{\sum }_{1\le k<k^{\prime }\le K}{\sum }_{i=1}^n{\sum }_{t\in I_i}cosDist(Z_{{\ell }_{target},i,t}\left(z_k\right),Z_{{\ell }_{target},i,t}\left(z_{k^{\prime }}\right))$

But this form of the objective makes it more transparent that a natural solution is to make each steering vector turn the output into gibberish (unless the LM latent space treats all gibberish alike, which I admit is possible). So maybe we would want a tunable term which encourages staying close to the unsteered activations, while staying far from the other steered activations. 
${max}_{z_1,\dots ,z_n}{\sum }_{1\le k<k^{\prime }\le K}{\sum }_{i=1}^n{\sum }_{t\in I_i}cosDist(Z_{{\ell }_{target},i,t}\left(z_k\right),Z_{{\ell }_{target},i,t}\left(z_{k^{\prime }}\right))-\lambda {\sum }_{i=1}^{K}D\left(z_k\right)$

Lastly, I would be interested in seeing the final output probability distribution over tokens instead of ${\ell }_{target}$ using KL for the distance, since in that domain we can extract very fine grained information from the model's activations. Let $D^{kl}\left(z\right)={\sum }_{i=1}^n{\sum }_{t\in I_i}KL(Z_{{\ell }_{unembed},i,t}\left(z\right)||Z_{{\ell }_{unembed},i,t}\left(0\right))$ in

${max}_{z_1,\dots ,z_n}{\sum }_{k=1}^K{\sum }_{k^{\prime }=1}^K{\sum }_{i=1}^n{\sum }_{t\in I_i}KL(Z_{{\ell }_{unembed},i,t}\left(z_k\right)||Z_{{\ell }_{unembed},i,t}\left(z_{k^{\prime }}\right))-\lambda {\sum }_{i=1}^K{D}^{kl}\left(z_k\right)$

Very interesting! I'm excited to read your post.

I take back the part about pi and update determining the causal structure, because many causal diagrams are constant with the same poly diagram

I think what is going on here is that both ${\nabla }^{\ast }$ and $G$ are of the form $\left(e^{\wedge }\right)\circ g\circ \ln$ with $g=\nabla$ and $g=E$, respectively. Let's define the star operator as $g^{\ast }=\left(e^{\wedge }\right)\circ g\circ \ln$. Then $(f\circ g{)}^{\ast }=(e^{\wedge })\circ (f\circ g)\circ \ln =(e^{\wedge })\circ f\circ \ln \circ (e^{\wedge })\circ g\circ \ln =f^{\ast }\circ g^{\ast }$, by associativity of function composition. Further, if $f$ and $g$ commute, then so do $f^{\ast }$ and $g^{\ast }$: $g^{\ast }\circ f^{\ast }=(g\circ f{)}^{\ast }=(f\circ g{)}^{\ast }=f^{\ast }\circ g^{\ast }.$

So the commutativity of the geometric expectation and derivative fall directly out of their representation as $E^{\ast }$ and ${\nabla }^{\ast }$, respectively, by commutativity of $E$ and $\nabla$, as long as they are over different variables. 

We can also derive what happens when the expectation and gradient are over the same variables: $({\nabla }_{\theta }\circ E_{x\sim P_{\theta }\left(x\right)}{)}^{\ast }$. First, notice that $(\ast k{)}^{\ast }(x)=e^{k\ast \ln x}=e^{\ln x\ast k}=x^k$, so $(\ast k{)}^{\ast }={(}^{\wedge }k)$.. Also $(+k{)}^{\ast }(x)=e^{k+\ln \left(x\right)}=e^k{e}^{\ln \left(x\right)}=x{e}^k⟹(+k{)}^{\ast }=(\ast e^k)$.

Now let's expand the composition of the gradient and expectation. $({\nabla }_{\theta }\circ E_{x\sim P_{\theta }\left(x\right)})\left(f\right(x\left)\right)={\nabla }_{\theta }\int P_{\theta }\left(x\right)f\left(x\right)dx=E_{x\sim P_{\theta }\left(x\right)}\left[{\nabla }_{\theta }\right(f\left(x\right)\ln P_{\theta }\left(x\right)\left)\right]$, using the log-derivative trick. So ${\nabla }_{\theta }\circ E_{x\sim P_{\theta }\left(x\right)}=E_{x\sim P_{\theta }\left(x\right)}\circ {\nabla }_{\theta }\circ (\ast \ln P_{\theta }(x\left)\right)$. 

Therefore, ${\nabla }_{\theta }^{\ast }\circ G_{x\sim P_{\theta }\left(x\right)}=({\nabla }_{\theta }\circ E_{x\sim P_{\theta }\left(x\right)}{)}^{\ast }$ $=E_{x\sim P_{\theta }\left(x\right)}^{\ast }\circ {\nabla }_{\theta }^{\ast }\circ (\ast \ln P_{\theta }(x){)}^{\ast }$ $=G_{x\sim P_{\theta }}\circ {\nabla }_{\theta }^{\ast }\circ {(}^{\wedge }\ln P_{\theta })$.

Writing it out, we have ${\nabla }_{\theta }^{\ast }{G}_{x\sim P_{\theta }\left(x\right)}\left[f\right(x\left)\right]=G_{x\sim P_{\theta }\left(x\right)}\left[{\nabla }_{\theta }^{\ast }\right(f\left(x{)}^{\ln P_{\theta }\left(x\right)}\right]$.

And if I pushed around symbols correctly, the geometric derivative can be pulled inside of a geometric expectation (${\nabla }_{\theta }^{\ast }{G}_{x\sim P\left(x\right)}\left[f\right(x\left)\right]=G_{x\sim P\left(x\right)}\left[{\nabla }_{\theta }^{\ast }f\right(x\left)\right]$) similarly to how an additive derivative can be pulled inside an additive expectation (${\nabla }_{\theta }{E}_{x\sim P\left(x\right)}\left[f_{\theta }\right(x\left)\right]=E_{x\sim P\left(x\right)}\left[{\nabla }_{\theta }{f}_{\theta }\right(x\left)\right]$). Also, just as additive expectation distributes over addition ($E\left[f\right(x)+g(x\left)\right]=E\left[f\right(x\left)\right]+E\left[g\right(x\left)\right]$), geometric expectation distributes over multiplication ($G\left[f\right(x\left)g\right(x\left)\right]=G\left[f\right(x\left)\right]G\left[g\right(x\left)\right]$).

If I try to use this framework to express two agents communicating, I get an image with a V1, A1, P1, V2, A2, and P2, with cross arrows from A1 to P2 and A2 to P1. This admits many ways to get a roundtrip message. We could have A1 -> P2 -> A2 -> P2 directly, or A1 -> P2 -> V2 -> A2 -> P1, or many cycles among P2, V2, and A2 before P1 receives a message. But in none of these could I hope to get a response in one time step the way I would if both agents simultaneously took an action, and then simultaneously read from their inputs and their current state to get their next state. So I have this feeling that pi : S -> Action and update : Observation x S -> S already bake in this active/passive distinction by virtue of the type signature, and this framing is maybe just taking away the computational teeth/specificity. And I can write the same infiltration and exfiltration formulas by substituting S_t for V_t, Obs_t for P_t, Action_t for A_t, and S_env_t for E_t.

Actually maybe this family is more relevant:
https://en.wikipedia.org/wiki/Generalized_mean, where the geometric mean is the limit as we approach zero.

The "harmonic integral" would be the inverse of integral of the inverse of a function -- https://math.stackexchange.com/questions/2408012/harmonic-integral