They are orthogonal to the "scale is all you need" people, and the "scale is all you need" thesis makes life the hardest for alignment/interpretability
[this is also how to get into neglectedness again, which EA adopted as a principle but recently forgot]
from Charles Rosenbauer:
This is neat, but this does little to nothing to optimize non-AI compute. Modern CPUs are insanely wasteful with transistors, plenty of room for multiple orders of magnitude of optimization there. This is only a fraction of the future of physics-optimized compute.