There's something called the No Free Lunch theorem which says, approximately, that there's no truly general algorithm for learning: if an algorithm predicts some environment better than chance, there must exist some adversarial environment on which it will do at least that much worse than chance. (Yes, this is even...
Epistemic status: slightly pruned brainstorming. I've been reading John Wentworth's posts on the natural abstraction hypothesis recently, and wanted to add some of my own thoughts. These are very much "directions to look in" rather than completed research, but I wanted to share some of the insights central to my...
Epistemic status: speculative. First, a few paper titles: * Pretrained Transformers as Universal Computation Engines * Can Wikipedia Help Offline Reinforcement Learning? (Answer: yes.) * Pretrained Transformers Improve Out-of-Distribution Robustness * Data Distributional Properties Drive Emergent In-Context Learning in Transformers The gist of the first three studies is that transformers...
There's something called the No Free Lunch theorem which says, approximately, that there's no truly general algorithm for learning: if an algorithm predicts some environment better than chance, there must exist some adversarial environment on which it will do at least that much worse than chance. (Yes, this is even true of Solomonoff induction.)
In the real world, this is almost completely irrelevant; empirically, general intelligence exists. However, leaving anthropics aside for a moment, we ought to find this irrelevance surprising in some sense; a robust theory of learning first needs to answer the question of why, in our particular universe, it's possible to learn anything at all.
I suspect that Wentworth's Telephone Theorem,... (read more)
While I like his idea, it's still not computable.
The notion of intelligence as compression is an old one; I believe Marcus Hutter was the first to formalize it back in the early 2000s (this is also where AIXI comes from). The problem with Hutter's formalism is that his definition of compressibility (Kolmogorov complexity) is uncomputable; "find the shortest Turing machine that outputs X" requires unbounded resources even if you have a halting oracle.
I believe that, in this paradigm, the NAH is fundamentally saying: well, for "natural" data, compressibility is computable; there's some minimal representation to which any sufficiently powerful (yet still finite) model will converge. The problem is, therefore, to figure out what a sufficiently powerful model looks like.
Epistemic status: slightly pruned brainstorming.
I've been reading John Wentworth's posts on the natural abstraction hypothesis recently, and wanted to add some of my own thoughts. These are very much "directions to look in" rather than completed research, but I wanted to share some of the insights central to my model of the NAH, which might be useful for others trying to expand on or simply grok Wentworth's work.
Epistemic status: speculative.
First, a few paper titles:
The gist of the first three studies is that transformers (specifically) trained on natural language (specifically) generalize better than expected, with little or no fine-tuning, not only to unseen tasks but even to unseen and apparently unrelated modalities like offline reinforcement learning. The last study takes this a step further—it doesn't actually pretrain on language at all, but instead tries to mimic the specific statistical properties of natural language that lead to this behavior with various sampling procedures from an image classification... (read 358 more words →)
Predictions over the course of my reading:
Guessed "dath ilan" from title.
Switched prediction to "Amenta" at mention of red hair.
Switched back to "dath ilan" (with much higher certainty) at the first use of capital-C "Civilization".