I mostly regard LLMs = [scaling a feedforward network on large numbers of GPUs and data] as a single innovation.

We never got around to write more unfortunately.

I recommend this paper for a good overview of compMecb https://arxiv.org/abs/cond-mat/9907176

Crypticity, Reverse Epsilon Machines and the Arrow of Time?

[see https://arxiv.org/abs/0902.1209 ]

Our subjective experience of the arrow of time is occasionally suggested to be an essentially entropic phenomenon. 

This sounds cool and deep but crashes headlong into the issue that the entropy rate and the excess entropy of any stochastic process is time-symmetric. I find it amusing that despite hearing this idea often from physicists and the like apparently this rather elementary fact has not prevented their storycrafting. 

Luckily, computational mechanics provides us with a measure that is not time symmetric: the stochastic complexity of the epsilon machine $C$

For any stochastic process we may also consider the epsilon machine of the reverse process, in other words the machine that predicts the past based on the future. This can be a completely different machine whose reverse stochastic complexity $C^{rev}$ is not equal to $C$. 

Some processes are easier to predict forward than backward. For example, there is considerable evidence that language is such a process. If the stochastic complexity and the reverse stochastic complexity differ we speak of a causally assymetric process. 

Alec Boyd pointed out to me that the classic example of a glass falling of a table is naturally thought of in these terms. The forward process is easy to describe while the backward process is hard to describe where easy and hard are meant in the sense of stochastic complexity: bits needed to specify the states of perfect minimal predictor, respectively retrodictor. 

rk. note that time assymmetry is a fundamentally stochastic phenomenon. THe underlyiing (let's say classicially deterministic) laws are still time symmetric. 

The hypothesis is then: many, most macroscopic processes of interest to humans, including other agents are fundamentally such causally assymetric (and cryptic) processes. 

The cases of the Dutch republic and English development after the Civil war are highly confounded. 
I wouldn't put too much stock in this kind of correlation. 

I think I speak for many when I ask you to please elaborate on this!

What did Yudkoswky get right?

• The central problem of AI alignment. I am not aware of anything in subsequent work that is not already implicit in Yudkowsky's writing.
• Short timelines avant le lettre. Yudkowsky was predicting AGI in his lifetime from the very start when most academics, observers, AI scientists, etc considered AGI a fairytale.
• Inherent and irreducible uncertainty of forecasting, foolishness of precise predictions. 
• The importance of (Pearlian) causality, Solomonoff Induction as theory of formal epistemology, Bayesian statistics, (Shannon) information theory, decision theory [especially UDT-shaped things].  
• (?nanotech, ?cryonics)
• if you had a timemachine to go back to 2010 you should buy bitcoin and write Harry Potter fanfiction

Preach, brother.

One hundred twenty percent agreed. Hubris is the downfall of the rationalist project.

I'm usually a skeptic of the usefulness of this kind of speculation but I found this a good read. I am particularly intrigued hy the suggestion of decomposability of goals.

Excited to see this go live, Nick!

Played around with Pantheon for a couple minutes. I wrote a couple of lines but I didn't get any daemons yet. How long should I have to wait for them to pop up?

On the word 'theory'. 

The word 'theory' is oft used and abused.

there is two ways 'theory' is used that are different and often lead to confusion. 

Theory in thescientific sense
the way a physicist would use: it's a model of the world that is either right or wrong. there might be competing theories and we neeed to have empirical evidence to figure out which one's right. Ideally, they agree with empirical evidence or at least are highly falsifiable. Importantly, if two theories are to conflict they need to actually speak about the same variables, the same set of measurable quantities.

Theory in the mathematician' sense; a formal framework
There is a related but different notion of theory that a mathematician would use: a theory of groups, of differential equations, of randomness, of complex systems, of etc etc. This is more like a formal framework for a certain phenomenon or domain.
It defines what the quantities, variables, features one is interested in even are. 

One often hears the question whether this (mathematical) theory makes testable predictions. This sounds sensible but doesn't really makes sense. It is akin to asking whether arithmetic or calculus makes testable predictions.* 

Theories in the mathematician's sense can't really be wrong or right since (at least in theory) everything is proven. Of course, theories in this sense can fail to say much about the real world, they might bake in unrealistic assumptions of course etc. 

Other uses of 'Theory'

The world 'theory' is also used in other disciplines. For instance, in literature studies where it is a denotes free form vacuous verbiage;  or in ML where 'theory' it is used for uninformed speculation. 

*one could argue that the theory of Peano Arithmetic actually does make predictions about natural numbers in the scientific sense, and more generally theories in the mathematical sense in a deep sense really are theories in the scientific sense.  I think there is something to this but 1. it hasn't been developed yet 2. mostly irrelevant in the present context.