jdp's Shortform

jdp

This is a special post for quick takes by jdp. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

FWIW I suspect the brain has fewer inductive biases than it looks like it does when you try to reverse engineer it from lesion studies.

If you did lesion studies on GPT it would look like it's made from a lot more parts than it is.

What are you picturing a "lesion study on GPT" looking like? Naively I imagine something like "train an SAE on the activations at some layer, then determine how often features activate together and then turn that into a distance metric and do clustering/dimensionality reduction, then ablate clusters of features and see how behavior changes". But I don't know that I'd particularly expect that to show the GPT as being made of many more parts than I actually think said GPT is made of. But also I don't have a super clear mental model of how many "parts" a GPT ... (read more)

jdp's Shortform

6

jdp's Shortform

6