Lee Smolin's book The Trouble with Physics: The Rise of String Theory, the Fall of a Science, and What Comes Next is ostensibly about why string theory can't solve what he calls the Five Great Problems in theoretical physics:
- "Combine general relativity and quantum theory into a single theory that can claim to be the complete theory of nature" i.e. "the problem of quantum gravity".
- "Resolve the problems in the foundations of quantum mechanics, either by making sense of the theory as it stands or by inventing a new theory that does make sense."
- "Determine whether or not the various particles and forces can be unified in a theory that explains them all as manifestations of a single, fundamental entity."
- "Explain how the values of the free constants in the standard model of particle physics are chosen in nature."
- "Explain dark matter and dark energy. Or, if they don't exist, determine how and why gravity is modified on large scales. More generally, explain why the constants of the standard model of cosmology, including the dark energy, have the values they do."
Actually, The Trouble with Physics is about a much broader problem—disruptive innovation as described in Clayton Christenson's The Innovator's Dilemma and Thomas Kuhn's The Structure of Scientific Revolutions. In Smolin's view, the scientific establishment is good at making small iterations to existing theories and bad at creating radically new theories. It's therefore not implausible that the solution to quantum gravity could come from a decade of solitary amateur work by someone totally outside the scientific establishment.
Extraordinary claims demand extraordinary evidence.
He [Carlo Rovelli] got so many e-mails [from string theorists] asserting that Mandelstam had proved [String Theory] finite that he decided to write to Mandelstam himself and ask his view. Mandelstam is retired, but he responded quickly. He explained that what he had proved is that a certain kind of infinite term does not appear anywhere in the theory. But he told us that he had not actually proved that the theory itself was finite, because other kinds of infinite terms might appear. No such term has ever been seen in any calculation done so far, but neither has anyone proved that one couldn't appear.
Smolin's book is full of evidence like this. I find his argument convincing because it aligns with my personal experience. I earned a bachelor's degree in physics because I wanted to help figure out the Great Problems. I wanted to discuss big ideas with Einsteins, Feynmans and Hawkings. In their place I encountered narrowly specialized postdocs. These PhDs are good at math but tend to have little education in the broader history of science. To them, the physical laws might as well have been handed down on stone tablets. Physicists' job is that of a madrasa.
This might be tenable if the foundations of physics (general relativity and quantum theory) were plausibly true. But general relativity and quantum theory contradict each other. They cannot both be correct. Therefore at least half of physics is wrong.
The scientific establishment isn't structured to resolve problems of this magnitude. Until we restructure the institution of physics so that it promotes diversity of thought (unlikely anytime soon) it's not inconceivable that the answers to the Five Great Problems could come from an amateur.
Smolin's book has inspired me to begin working on a theory of quantum gravity. I'll need to learn new things like quantum field theory. I might give up before getting anywhere. But at least I know I don't understand basic physics. That puts me in good company.
I think I can safely say that nobody understands quantum mechanics.
― Richard Feynman
Update: I'm working on AI instead.