This is a special post for quick takes by TurnTrout. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Idea: Speed up ACDC by batching edge-checks. The intuition is that most circuits will have short paths through the transformer, because Residual Networks Behave Like Ensembles of Relatively Shallow Networks (https://arxiv.org/pdf/1605.06431.pdf). Most edges won't be in most circuits. Therefore, if you're checking KL of random-resampling edges e1 and e2, there's only a small chance that e1 and e2 interact with each other in a way important for the circuit you're trying to find. So statistically, you can check eg e1, ... e100 in a batch, and maybe ablate 95 edges all at once (due to their individual KLs being low).
If true, this predicts that given a threshold T, and for most circuit subgraphs H of the full network graph G, and for the vast majority of e1, e2 in H:
KL(G || H \ {e2} ) - KL(G || H) < T iff KL(G || H \ {e2, e1} ) - KL(G || H \ {e1}) < T
(That is, e1's inclusion doesn't change your pruning decision on e2)
Neel Nanda suggests that it's particularly natural to batch edges in the same layer.
I do not understand the appeal of MOJO. If you're going to overcomplicate Python to the point of C++, just use C++. Or, you know, a saner systems language like Rust. CPython has C interop already.
For what it's worth, I've had to drop from python to C# on occasion for some bottlenecks. In one case, my C# implementation was 418,000 times faster than the python version. That's a comparison between a poor python implementation and a vectorized C# implementation, but... yeah.
3Thomas Kwa
Adria (other coauthor on the paper) told me that Redwood ported ACDC to Rust early on, which did provide a useful speedup (idk how much but my guess is 10-100x?) but made it harder to maintain. I'm currently working in Python. I wouldn't believe those marketing numbers for anything but the Mandelbrot task they test it on, which has particularly high interpreter overhead.
The bigger problem with ACDC is it doesn't use gradients. Attribution patching fixes this and gets >100x speedups, and there should be even better methods. I don't expect circuit discovery to be usefully ported to a fast language again, until it is used in production.