E.G. Blee-Goldman's Shortform

21st May 2025

1 min read

2

This is a special post for quick takes by E.G. Blee-Goldman. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

E.G. Blee-Goldman's Shortform

10 comments, sorted by

top scoring

Click to highlight new comments since: Today at 9:53 AM

[-]E.G. Blee-Goldman5mo150

Jack Morris has posted this thread https://x.com/jxmnop/status/1925224612872233081 about his paper "Harnessing the Universal Geometry of Embeddings"

Have others thought through what this means for the notion of fundamentally alien internal ontologies? Would love any ideas! Sorry if missed a post on it.

[-]cubefox5mo50

Thanks for this reference. It arguably means aliens don't have alien ontologies. Previous related discussion.

[-]Evan R. Murphy5mo30

Let me know if anyone has thoughts on this question I just posted as well: Does the Universal Geometry of Embeddings paper have big implications for interpretability?

[-]cubefox5mo30

Is this evidence for the natural abstraction hypothesis @johnswentworth?

[-]johnswentworth5mo92

My default assumption on all empirical ML papers is that the authors Are Not Measuring What They Think They Are Measuring.

It is evidence for the natural abstraction hypothesis in the technical sense that P[NAH|paper] is greater than P[NAH], but in practice that's just not a very good way to think about "X is evidence for Y", at least when updating on published results. The right way to think about this is "it's probably irrelevant".

[-]E.G. Blee-Goldman5mo10

Thank you John! Is there an high-bit or confounder controlling evidence that would move your prior? Say something like english + some other language? (Also I might be missing something deeper about the heuristic in general, if so I apologize!)

[-]E.G. Blee-Goldman3mo21

I found this paper by Amir Zur and others really interesting: It's Owl in the Numbers:
Token Entanglement in Subliminal Learning where they try to explain subliminal learning (the notion that "language model fine-tuned on seemingly meaningless data from a teacher model acquires the teacher's hidden behaviors.")

The researchers found that certain concepts like "owl" and "087" can become entangled during training (the probability of one increases the probability of the other.)

Fascinating and would be curious to hear what others think!