Haven't seen it discussed in the context of ASI. I genuiely believe such "controlled AI catastrophe" may be the best strategy for an advanced civilization to survive past ASI. I know this raises difficult ethical questions so would love to be proven wrong.

One would hope that progress in interpretability will allow us to do much more refined "mind control" than adding difference vectors.

As a European worried about ASI, it's really not clear to me that China would be more reckless and unsafe that the US. In fact, I don't really see how this is possible, the US seems already maximally reckless as far as I can see. I feel that this war mongering US v China is what is really dangerous, I don't see a hope for humanity if we cannot all unite against ASI.

You are entitled to your opinion but I'm a tenured theoretical physicist and the view I expressed is not controversial and shared by most people in the field. Most theoretical physicists are really scared or excited by AI. I believe it is an almost consensus in the community that AI will eventually "solve physics", and many believe that this could happen fairly soon.

The progress in fundamental physics has always been towards increasing simplicity and this is not going to change. The universe is an incredibly constrained system and it is almost impossible to come up with a consistent unifying model (we are still looking for it). We already have more than enough data for a sufficiently intelligent entity to figure out "the theory of everything", it is only a lack of imagination.

Physics is not something that you solve by “acquiring more data”. The underlying laws are incredibly simple. Famously Einstein derived both the special and general theory of relativity almost only from pure thought. Experimental data is needed as confirmation. As a theoretical physicist myself, I have no doubt that ASI will be able to essentially figure out “all of physics”. 

Agree with everything but I still believe that developing interpretability is our best realistic shot at survival.

Thanks for this post. I am theoretical physicist and I agree that LLM interpretability is a physics problem. I think more and more physicists are becoming interested in this problem because it is obvious to us that physics methods, ideas and tools will be useful here. The whole field of physics is just mechanistic interpretability of the universe. Physics is quite broad and there are no fixed rules. The only goal is to increase understanding in any possible way and in fact one should always change and redefine the rules of the game. LLMs (as a toy model of intelligence) is just another complex system and one with which it is very easy to experiment.

Physics is incredibly precise (only pure mathematics is more rigorous) and everything does indeed follows from a small number of basic principles. These principles are not sacrosent though, more like useful assumptions, and in quantum gravity we don't expect locality to strictly holds. The solution of the black hole information paradox mentioned above is precisely showing that the late Hawking radiation (which is very far from the black hole) is the same quantum system as the black hole interior. This is a manifestion of quantum entanglement which is realized in gravity through a connected geometry (wormhole). So the solution of the Hawking information paradox is that the information thrown into a black hole is not lost after evaporation, it has escaped into Hawking radiation through tiny quantum wormholes. These non-local effects only happen at the Planck scale or in comlplicated systems such as old black holes. 

About spin-statistics, the modern understanding is that is just follows from the representation theory of the Lorentz group. The existence of fermions is due to the fact that SO(3,1) is not simply connected so what gets represented is its universal cover (a double cover) so there exists fermionic representations with anti-commuting properties. Their properties are fixed from the requirement that they must form consistent unitary representations (in order to have a quantum theory giving positive probabilities). As you mention, in lower dimensions, anyons exist, and this is due to the fact that the lower-dimensional Lorentz groups have different properties, their universal cover can be an infinite cover so there are anyonic representations too.

String theorist here. Progress has absolutely not stalled. The last big theoretical breakthrough was the AdS/CFT correspondence ( https://arxiv.org/abs/hep-th/9711200  ). This was our Transformers moment. This paper revolutioned the field, has now 20k+ citations, the most for any physics theory paper. It did completely solve the problem of quantum gravity for AdS spacetimes (negative curvature). There are too many things that have followed from this but I can give a few tidbits. For example it was understood that quantum black holes are the most chaotic objects in nature (https://arxiv.org/abs/1306.0622 https://arxiv.org/abs/1503.01409). A deep relationship between quantum gravity and quantum information theory was found (e.g, https://arxiv.org/abs/hep-th/0603001 https://arxiv.org/abs/1306.0533  https://arxiv.org/abs/1411.7041 ) which ultimately led recently to a solution of the black hole information paradox 50+ years after posed by Hawking! (https://arxiv.org/abs/2006.06872). There are caveats ofc, our universe is a dS spacetime (positive curvature) so current work is trying to extend AdS/CFT to cosmological spacetimes like our own.

The main reason string theory has bad press in the public is due to the fact that most top scientits don't have time to do public outreach (and their time is probably spent better elsewhere). So the only "physicists" who do public outreach are the ones who have nothing better to do... they are often bitter or disgrnuntled people whose subfield is becoming irrelevant and who have fallen behind. So they maintain blogs or publish books to feel more relevant saying things like "not even wrong" "physics is dead" "lost in math" etc. (there are exceptions ofc such as Brian Greene, Max Tegmark etc.).