leogao's Shortform

leogao

This is a special post for quick takes by leogao. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

metrics for pragmatic interp methods

one intuition pump i like: suppose an alien showed up and claimed to fully understand GPT-5, but alas, the details are far too complicated to fit in a human brain. what questions could you ask the alien in order to prove that it truly does have some kind of insight?

ideally, a good question would be something that is:

very costly to answer with only black box access,
vastly easier with mechanistic understanding,
useful to answer even up to superintelligence scale,
and relatively cheap to check the correctness of.

my current best guess for a metric that satisfies all of these criteria is jailbreak discovery (and more generally, whether you can discover inputs that produce rare outputs). see https://www.lesswrong.com/posts/YiRsCfkJ2ERGpRpen/leogao-s-shortform?commentId=guGHxetQQCEnef7k9 for related discussion

2Daniel Tan2mo

after some more thought, my guess is that the method itself might not scale very far if you ban "magic ood generalization" but (i) they will definitely explore the problem in more depth and (ii) they might come up with lots more empirical settings / metrics that capture the spirit of what you want

my take is you need to either eliminate magic ood generalization or specifically study the magic ood generalization really carefully, like actually do science on it, not that typical ML paper stuff where you show that it "works sometimes i guess"

4JennaS2mo

Jail breaking does not seem exceptionally difficult. Pliny and a swarm of agents do it routinely. He even developed a universal one.

leogao's Shortform

8

Ω 4

leogao's Shortform

8

Ω 4