(Fwiw, I don’t remember problems with stipend payout at seri mats in the winter program. I was a winter scholar 2022/23.)

This is very helpful, thanks! Actually, the post includes several sections, including in the appendix, that might be more interesting to many readers than the grant recommendations themselves. Maybe it would be good to change the title a bit so that people also expect other updates.

Thanks for the reply!

As I show in the examples in DSLT1, having degenerate Fisher information (i.e. degenerate Hessian at zeroes) comes in two essential flavours: having rank-deficiency, and having vanishing second-derivative (i.e. $K\left(w\right)=w^4$). Precisely, suppose $d$ is the number of parameters, then you are in the regular case if $K\left(w\right)$ can be expressed as a full-rank quadratic form near each singularity, 

$$K\left(w\right)=\sum _{i=1}^d{w}_i^2.$$

Anything less than this is a strictly singular case. 

So if $K\left(w\right)=w^2$, then $w=0$ is a singularity but not a strict singularity, do you agree? It still feels like somewhat bad terminology to me, but maybe it's justified from the algebraic-geometry--perspective. 

Zeta Functions in Singular Learning Theory

In this shortform, I very briefly explain my understanding of how zeta functions play a role in the derivation of the free energy in singular learning theory. This is entirely based on slide 14 of the SLT low 4 talk of the recent summit on SLT and Alignment, so feel free to ignore this shortform and simply watch the video.

The story is this: we have a prior $\phi \left(w\right)$, a model $p(x\mid w)$, and there is an unknown true distribution $q\left(x\right)$. For model selection, we are interested in the evidence of our model for a data set $D_n=\{x_1,\dots ,x_n\}$, which is given by

where $K_n\left(w\right)=\frac{1}{n}{\sum }_{i=1}^n\log \frac{q\left(x_i\right)}{p(x_i\mid w)}$ is the empirical KL divergence. In fact, we are interested in selecting the model that maximizes the average of this quantity over all data sets. The average is then given by

where $K\left(w\right)={\int }_{x\sim q}q\left(x\right)\log \frac{q\left(x\right)}{p(x\mid w)}$ is the Kullback-Leibler divergence. 

But now we have a problem: how do we compute this integral? Computing this integral is what the free energy formula is about. 

The answer: by computing a different integral. So now, I'll explain the connection to different integrals we can draw. 

Let

which is called the state density function. Here, $\delta$ is the Dirac delta function.  For different $t$, it measures the density of states (= parameter vectors) that have $K\left(w\right)=t$. It is thus a measure for the "size" of different level sets. This state density function is connected to two different things. 

Laplace Transform to the Evidence

First of all, it is connected to the evidence above. Namely, let $L\left(v\right)$ be the Laplace transform of $v$. It is a function $L\left(v\right):N\to R$ given by

In first step, we changed the order of integration, and in the second step we used the defining property of the Dirac delta. Great, so this tells us that $L\left(v\right)=\overline{Z}$! So this means we essentially just need to understand $v$.

Mellin Transform to the Zeta Function

But how do we compute $v$? By using another transform. Let $M\left(v\right)$ be the Mellin transform of $v$. It is a function $M\left(v\right):C\to C$ (or maybe only defined on part of $C$?) given by

Again, we used a change in the order of integration and then the defining property of the Dirac delta. This is called a Zeta function. 

What's this useful for?

The Mellin transform has an inverse. Thus, if we can compute the zeta function, we can also compute the original evidence as

Thus, we essentially changed our problem to the problem of studying the zeta function $M\left(v\right).$ To compute the integral of the zeta function, it is then useful to perform blowups to resolve the singularities in the set of minima of $K\left(w\right)$, which is where algebraic geometry enters the picture. For more on all of this, I refer, again, to the excellent SLT low 4 talk of the recent summit on singular learning theory. 

Thanks for the answer! I think my first question was confused because I didn't realize you were talking about local free energies instead of the global one :) 

As discussed in the comment in your DSLT1 question, they are both singularities of $K\left(w\right)$ since they are both critical points (local minima).

Oh, I actually may have missed that aspect of your answer back then. I'm confused by that: in algebraic geometry, the zero's of a set of polynomials are not necessarily already singularities. E.g., in $f(x,y)=xy$, the zero set consists of the two axes, which form an algebraic variety, but only at $(0,0)$ is there a singularity because the derivative disappears.
Now, for the KL-divergence, the situation seems more extreme: The zero's are also, at the same time, the minima of $K$, and thus, the derivative disappears at every point in the set $W_0$. This suggests every point in $W_0$ is singular. Is this correct?

So far, I thought "being singular" means the effective number of parameters around the singularity is lower than the full number of parameters. Also, I thought that it's about the rank of the Hessian, not the vanishing of the derivative. Both perspectives contradict the interpretation in the preceding paragraph, which leaves me confused. 

The uninteresting answer is that SLT doesn't care about the prior (other than its regularity conditions) since it is irrelevant in the $n\to \infty$ limit.

I vaguely remember that there is a part in the MDL book by Grünwald where he explains how using a good prior such as Jeffrey's prior somewhat changes asymptotic behavior for $n\to \infty$, but I'm not certain of that. 

Thanks also for this post! I enjoy reading the sequence and look forward to post 5 on the connections to alignment :) 

At some critical value $\theta ={\theta }_c$, we recognise a phase transition as being a discontinuous change in the free energy or one of its derivatives, for example the generalisation error $G_n=E\left[F_{n+1}\right]-E\left[F_n\right]$.

"Discontinuity" might suggest that this happens fast. Yet, e.g. in work on grokking, it actually turns out that these "sudden changes" happen over a majority of the training time (often, the x-axis is on a logarithmic scale). Is this compatible, or would this suggest that phenomena like grokking aren't related to the phase transitions predicted by SLT?

There is, however, one fundamentally different kind of "phase transition" that we cannot explain easily with SLT: a phase transition of SGD in time, i.e. the number gradient descent steps. The Bayesian framework of SLT does not really allow one to speak of time - the closest quantity is the number of datapoints $n$, but these are not equivalent. We leave this gap as one of the fundamental open questions of relating SLT to current deep learning practice.

As far as I know, modern transformers are often only trained once on each data sample, which should close the gap between SGD time and the number of data samples quite a bit. Do you agree with that perspective?

In general, it seems to me that we're probably most interested in phase transitions that happen across SGD time or with more data samples, whereas phase transitions related to other hyperparameters (for example, varying the truth as in your examples here) are maybe less crucial. Would you agree with that?

Would you expect that most phase transitions in SGD time or the number of data samples are first-order transitions (as is the case when there is a loss-complexity tradeoff), or can you conceive of second-order phase transitions that might be relevant in that context as well?

Which altered the posterior geometry, but not that of $K\left(w\right)$ since $p\left(w|D_n\right)\approx e^{-nK\left(w\right)}$ (up to a normalisation factor).

I didn't understand this footnote. 

but the node-degeneracy and orientation-reversing symmetries only occur under precise configurations of the truth.

Hhm, I thought that these symmetries are about configurations of the parameter vector, irrespective of whether it is the "true" vector or not.
Are you maybe trying to say the following? The truth determines which parameter vectors are preferred by the free energy, e.g. those close to the truth. For some truths, we will have more symmetries around the truth, and thus lower RLCT for regions preferred by the posterior. 

We will use the label weight annihilation phase to refer to the configuration of nodes such that the weights all point into the centre region and annihilate one another.

It seems to me that in the other phase, the weights also annihilate each other, so the "non-weight annihilation phase" is a somewhat weird terminology. Or did I miss something?

The weight annihilation phase $E_{\text{NonWA}}$ is never preferred by the posterior

I think there is a typo and you meant $E_{\text{WA}}$.

Thanks Liam also for this nice post! The explanations were quite clear. 

The property of being singular is specific to a model class $f(x,w)$, regardless of the underlying truth.

This holds for singularities that come from symmetries where the model doesn't change. However, is it correct that we need the "underlying truth" to study symmetries that come from other degeneracies of the Fisher information matrix? After all, this matrix involves the true distribution in its definition. The same holds for the Hessian of the KL divergence. 

Both configurations, non-weight-annihilation (left) and weight-annihilation (right)

What do you mean with non-weight-annihilation here? Don't the weights annihilate in both pictures?

In particular, it is the singularities of these minimum-loss sets — points at which the tangent is ill-defined — that determine generalization performance.

To clarify: there is not necessarily a problem with the tangent, right? E.g., the function $f\left(x\right)=x^4$ has a singularity at $0$ because the second derivative vanishes there, but the tangent is define. I think for the same reason, some of the pictures may be misleading to some readers. 

• A model, $p\left(x|w\right)$, parametrized by weights $w\in W\subset R^d$, where $W$ is compact;

Why do we want compactness? Neural networks are parameterized in a non-compact set. (Though I guess usually, if things go well, the weights don't blow up. So in that sense it can maybe be modeled to be compact)

The empirical Kullback-Leibler divergence is just a rescaled and shifted version of the negative log likelihood.

I think it is only shifted, and not also rescaled, if I'm not missing something. 

But these predictions of "generalization error" are actually a contrived kind of theoretical device that isn't what we mean by "generalization error" in the typical ML setting.

Why is that? I.e., in what way is the generalization error different from what ML people care about? Because real ML models don't predict using an updated posterior over the parameter space? (I was just wondering if there is a different reason I'm missing)

Thanks for the answer mfar!

Yeah I remember also struggling to parse this statement when I first saw it. Liam answered but in case it's still not clear and/or someone doesn't want to follow up in Liam's thesis, $x$ is a free variable, and the condition is talking about linear dependence of functions of $x$.

Consider a toy example (not a real model) to help spell out the mathematical structure involved: Let $f(x,w)=(w_1+2w_2)x$ so that $\frac{\partial }{\partial w_1}f(x,w)=x$ and $\frac{\partial }{\partial w_2}f(x,w)=2x$. Then let $g$ and $h$ be functions such that $g\left(x\right)=x$ and $h\left(x\right)=2x$.. Then the set of functions $\{g,h\}$ is a linearly dependent set of functions because $h-2g=0$.

Thanks! Apparently the proof of the thing I was wondering about can be found in Lemma 3.4 in Liam's thesis. Also thanks for your other comments!

Thanks for the answer Liam! I especially liked the further context on the connection between Bayesian posteriors and SGD. Below a few more comments on some of your answers:

The partition function is equal to the model evidence $Z_n=p\left(D_n\right)$, yep. It isn’t equal to $p\left(\right(Y_i\left)|\right(X_i\left)\right),$(I assume $i$ is fixed here?) but is instead expressed in terms of the model likelihood and prior (and can simply be thought of as the “normalising constant” of the posterior), 

$$p\left(D_n\right)={\int }_W\phi \left(w\right)\prod _{i=1}^{n}p(y_i,x_i|w)dw$$

and then under this supervised learning setup where we know $q\left(x_i\right)$, we have $p(y_i,x_i|w)=p(y_i|x_i,w)q\left(x_i\right)$. Also note that this does “factor over $i$” (if I’m interpreting you correctly) since the data is independent and identically distributed.  

I think I still disagree. I think everything in these formulas needs to be conditioned on the $X$-part of the dataset. In particular, I think the notation $p\left(D_n\right)$ is slightly misleading, but maybe I'm missing something here.

I'll walk you through my reasoning: When I write $\left(X_i\right)$ or $\left(Y_i\right)$, I mean the whole vectors, e.g., $(X_i{)}_{i=1,\dots ,n}$. Then I think the posterior compuation works as follows:

That is just Bayes rule, conditioned on $\left(X_i\right)$ in every term. Then, $p(w\mid \left(X_i\right))=\phi \left(w\right)$ because from $X$alone you don't get any new information about the conditional $q(Y\mid X)$ (A more formal way to see this is to write down the Bayesian network of the model and to see that $w$ and $X_i$ are d-separated). Also, conditioned on $w$, $p$ is independent over data points, and so we obtain

So, comparing with your equations, we must have $Z_n=p(\left(Y_i\right)\mid \left(X_i\right)).$ Do you think this is correct?

Btw., I still don't think this "factors over $i$". I think that

$Z_n\ne {\prod }_{i=1}^{n}p(Y_i\mid X_i).$

The reason is that old data points should inform the parameter $w$, which should have an influence on future updates. I think the independence assumption only holds for the true distribution and the model conditioned on $w$. 

If you expand that term out you find that 

$$\begin{array}{cc}& {\int }_W(w-w_0{)}^T\frac{\partial \phi }{\partial w}{|}_{w=w_0}\exp (-(w-w_0{)}^{T}I(w_0\left)\right(w-w_0\left)\right)dw\\ & =\frac{\partial \phi }{\partial w}{|}_{w=w_0}{\int }_W(w-w_0{)}^T\exp (-(w-w_0{)}^{T}I(w_0\left)\right(w-w_0\left)\right)dw=0\end{array}$$

because the second integral is the first central moment of a Gaussian. The derivative of the prior is irrelevant. 

Right. that makes sense, thank you! (I think you missed a factor of $n/2$, but that doesn't change the conclusion)

Thanks also for the corrected volume formula, it makes sense now :)