I'm having trouble understanding your suggestion, especially the second paragraph.  Could you spell it out a bit more?

There are, but what does having a length below 10^90 have to do with the solomonoff prior? There's no upper bound on the length of programs.

Yes, you are missing something.

Any DEADCODE that can be added to a 1kb program can also be added to a 2kb program. The net effect is a wash, and you will end up with a $2^{1000}$ ratio over priors

Thanks for sharing that study. It looks like your team is already well-versed in this subject!

You wouldn't want something that's too hard to extract, but I think restricting yourself to a single encoder layer is too conservative - LLMs don't have to be able to fully extract the information from a layer in a single step.

I'd be curious to see how much closer a two-layer encoder would get to the ITO results.

:Here's my longer reply.

I'm extremely excited by the work in SAEs and their potential for interpretability, however I think there is a subtle misalignment in the SAE architecture and loss function, and the actual desired objective function.

The SAE loss function is:

$L(x;W_{dec},b_{dec})=E_x[||x-\hat{x}|{|}^2+\lambda ||f(x\left)|{|}_1\right]$, where $||f\left(x\right)|{|}_1={\sum }_i{f}_i\left(x\right)$ is the $\ell 1$-Norm.

or 

$L\left(x\right)=E_x[||x-W_{dec}f(x)-b_{dec}|{|}^2+\lambda ||f(x\left)|{|}_1\right]$

I would argue that, however, what you are actually trying to solve is the sparse coding problem:

$L(x;W_{dec},b_{dec})=E_x[{min}_f||x-W_{dec}f-b_{dec}|{|}^2+\lambda ||f|{|}_1]$

where, importantly, the inner optimization is solved separately (including at runtime).

Since$f$ is an overcomplete basis, finding $f^{\ast }$ that minimizes the inner loop (also known as basis pursuit denoising^[1] ) is a notoriously challenging problem, one which a single-layer encoder is underpowered to compute. The SAE's encoder thus introduces a significant error ${\tilde{f}}_{enc}$ , which means that you are actual loss function is:

$L(x;\Theta )=E_x[||x-W_{dec}(f^{\ast }+{\tilde{f}}_{enc})-b_{dec}|{|}^2+\lambda ||f^{\ast }+{\tilde{f}}_{enc}|{|}_1]$

The magnitude of the errors would have to be determined empirically, but I suspect that it is enough to be a significant source of error..

There are a few things you could do reduce the error:

1. Ensuring that $W_{dec}$ obeys the restricted isometry property^[2] (i.e. a cap on the cosine similarity of decoder weights), or barring that, adding a term to your loss function that at least minimizes the cosine similarities.
2. Adding extra layers to your encoder, so it's better at solving for $f^{\ast }$.
3. Empirical studies to see how large the feature error is / how much reconstruction error it is adding.

1. ^{^}

   https://epubs.siam.org/doi/abs/10.1137/S003614450037906X?casa_token=E-R-1D55k-wAAAAA:DB1SABlJH5NgtxkRlxpDc_4IOuJ4SjBm5-dLTeZd7J-pnTAA4VQQ2FJ6TfkRpZ3c93MNrpHddcI

2. ^{^}

   http://www.numdam.org/item/10.1016/j.crma.2008.03.014.pdf

This is great work. My recommendation: add a term in your loss function that penalizes features with high cosine similarity.

I think there is a strong theoretical underpinning for the results you are seeing.

I might try to reach out directly - some of my own academic work is directly relevant here.

This is one of those cases where it might be useful to list out all the pros and cons of taking the 8 courses in question, and then thinking hard about which benefits could be achieved by other means.

Key benefits of taking a course (vs. Independent study) beyond the signaling effect might include:

• precommitting to learning a certain body of knowledge
• curation of that body of knowledge by an experienced third party
• additional learning and insight from partnerships / teamwork / office hours

But these depend on the courses and your personality. The precommitment might be unnecessary due to your personal work habits, the curation might be misaligned with what you are interested in learning, and the other students or TAs may not have useful insights that you can't figure out in your own.

Hope that helps.

Instead of demanding orthogonal representations, just have them obey the restricted isometry property.

Basically, instead of requiring  $\forall i\ne j:<x_i,x_j>=0$, we just require $\forall i\ne j:x_i\cdot x_j\le ϵ$ .

This would allow a polynomial number of sparse shards while still allowing full recovery.

I think the success or failure of this model really depends on the nature and number of the factions. If interfactional competition gets too zero-sum (this might help us, but it helps them more, so we'll oppose it) then this just turns into stasis.

During ordinary times, vetocracy might be tolerable, but it will slowly degrade state capacity. During a crisis it can be fatal.

Even in America, we only see this factional veto in play in a subset of scenarios - legislation under divided government. Plenty of action at the executive level or in state governments don't have to worry about this.