this feels like a subtweet of our recent paper on circuit sparsity. I would have preferred a direct response to our paper (or any other specific paper/post/person), rather than a dialogue against a hypothetical interlocutor.
I think this post is unfairly dismissive of the idea that we can guess aspects of the true ontology and iterate empirically towards it. it makes it sound like if you have to guess a lot of things right about the true ontology before you can make any empirical progress at all. this is a reasonable view of the world, but I think evidence so far rules out the strongest possible version of this claim.
SAEs are basically making the guess that the true ontology should activate kinda sparsely. this is clearly not enough to pin down the true ontology, and obviously at some point activation sparsity stops being beneficial and starts hurting. but SAE features seem closer to the true ontology than the neurons are, even if they are imperfect. this should be surprising if you think that you need to be really correct about the true ontology before you can make any progress! making the activations sparse is this kind of crude intervention, and you can imagine a world where SAEs don't find anything interesting at all because it's much easier to just find random sparse garbage, and so you need more constraints before you pin down something even vaguely reasonable. but we clearly don't live in that world.
our circuit sparsity work adds another additional constraint: we also enforce that the interactions between features are sparse. (I think of the part where we accomplish this by training new models from scratch as an unfortunate side effect; it just happens to be the best way to enforce this constraint.) this is another kind of crude intervention, but our main finding is that this gets us again slightly closer to the true concepts; circuits that used to require a giant pile of SAE features connected in an ungodly way can now be expressed simply. this again seems to suggest that we have gotten closer to the true features.
if you believe in natural abstractions, then it should at least be worth trying to dig down this path and slowly add more constraints, seeing whether it makes the model nicer or less nice, and iterating.
fwiw, I think the 100-1000x number is quite pessimistic, in that we didn't try very hard to make our implementation efficient, we were entirely focused on making it work at all. while I think it's unlikely our method will ever reach parity with frontier training methods, it doesn't seem crazy that we could reduce the gap a lot.
and I think having something 100x behind the frontier (i.e one GPT worth) is still super valuable for developing a theory of intelligence! like I claim it would be super valuable if aliens landed and gave us an interpretable GPT-4 or even GPT-3 without telling us how to make our own or scale it up.
1I don't really care that much about not having a monitor. it's a minor productivity hit, whereas not having reliable vaguely-fast internet completely ruins productivity.
I would absolutely fly so much more. weekend trips become way more feasible if I can fly out on Friday and return on Monday. working remotely but visiting HQ occasionally (or otherwise splitting time between two cities) gets a lot easier, because you no longer lose a day of productivity (or a night of sleep) each time.
1last time I flew Delta it was not amazing, though to be fair I don't fly Delta very often. I generally fly United or JetBlue, both of which have a rep for "good" wifi, but I've never felt particularly satisfied by it.
I'd personally pay more, endure less convenient timing, and sit in a less comfortable seat if it meant I had fast wifi.
like right now flying is pretty time costly for me because most of my highest value work can only be done with internet, so flying means losing a lot of high productivity hours. fast wifi would mean the only time cost of flying is the tiny bit I spend walking through the airport on either end.
I think anyone who has ever tried to work on a plane knows that plane wifi is bad enough to reduce your productivity hugely. so I don't think business travellers who are already paying thousands to fly would shy away from paying hundred of dollars for actually good wifi on a long haul flight.
I'd predict most business travellers are not really using being on a plane as an excuse to not work.
why is it taking so long to upgrade planes to use starlink? it doesn't sound like there are huge technical barriers to doing so, and it would be hugely profitable. i would not only pay a lot per flight for good wifi, i would also fly way more often
people lie about some crazy shit
fwiw, I'm pessimistic that you will actually be able to make big compute efficiency improvements even by fully understanding gpt-n. or at least, for an equivalent amount of effort, you could have improved compute efficiency vastly more by just doing normal capabilities research. my general belief is that the kind of understanding you want for improving compute efficiency is at a different level of abstraction than the kind of understanding you want for getting a deep understanding of generalization properties.