I mostly regard LLMs = [scaling a feedforward network on large numbers of GPUs and data] as a single innovation.

I like this. 

Another explanation I have heard: 

a popular theory of aging is the mitochrondial theory of aging. 

There are several variants of this theory some of which are definitely false, while some are plausibly in sorta-the-direction. It's a big controversy and I'm not an expert yada yada yada. Let me assume something like the following is true: aging is a metabolic phenomena where mitochrondia degrade overtime and at some point start to leak damaging byproducts which is substantially responsible for aging. Mitochrondial DNA have less repair mechanism than nuclear DNA. Over time they accrue mutations that are bad (much quicker than nuclear dna). 

Species that reproduce fast & many may select less on (mitochrondial) mutational load since its matter less. On the other hand, species that have more selection on mitochrondial mutational load for whatever reason are less fecund. E.g. fetuses may be spontaneously aborted if the mitochrondia have too many mutations. 

Some pieces of evidence: eggs contain the mitochrondia and are 'kept on ice', i.e. they do not metabolize. Birds have a much stronger selection pressure for high-functioning metabolism (because of flight)^[1] and plausibly 'better mitochrondia'. 

[there are also variant-hypotheses possible that have a similar mutation meltdown story but don't go through mitochrondia per se. There is some evidence and counterevidence for epigenetic and non-mitochrondial mutational meltdown theories of againg too. So not implausible]

1. ^{^}

   compare bats? what about their lifespans?

Wow! This would be awesome!

I didn't look at this case in particular. In my experience, LLMs are quite good at regurgitating definitions from long math texts - which otherwise can take some effort to find by hand. 

The Virtue of Comparison Shopping 
Comparison shopping, informed in-depth reviewing, answering customer surveys plausibly have substantial positive externalities. It provides incentives through local actors, avoids preference falsification or social desirability bias, and is non-coercive & market-based. 

Plausibly it is even has a better social impact than many kinds of charitable donations or direct work. This is not so hard since it seems that the latter contains many kinds of interventions that have neglibible or even negative impact. 

Wow! This looks fantastic. 

I missed this the first time around - and judging from the number of upvotes so did a lot of other people. A shame. 

Here's to hoping more folks will stumble upon your sequence like I did. 

As a complete noob in all things mechinterp can somebody explain how this is not in conflict with SAE enjoyers saying they get reconstruction loss in the high 90s or even 100 %?

I understand the logscale argument that Lucius is making but still seems surprising ? Is this really what's going on or are they talking about different things here.

Thank you for doing this work. This is is a valuable piece of evidence.

SLT-enjoyers everywhere rejoice

This sounds genuinely worrying. Largest negative timeline update I've made in many months.