Physics seems to have a bunch of useful epistemic techniques which haven’t been made very legible yet.
The two big legible epistemic techniques in technical fields are Mathematical Proofs, and The Scientific Method. Either derive logically X from some widely-accepted axioms, or hypothesize X and then do a bunch of experiments which we’d expect to come out some other way if X were false. It seems pretty obvious that science requires a bunch of pieces besides those in order to actually work in practice, but those are the two which we’ve nailed down most thoroughly.
Then there’s less-legible methods. Things like fermi estimates, gears-level models, informal mathematical arguments, an aesthetic sense for kinds-of-models-which-tend-to-generalize-well, the habit of figuring out qualitative features of an answer before calculating it, back-of-the-envelope approximations, etc.
Take informal mathematical arguments, for example. We’re talking about things like the use of infinitesimals in early calculus, or delta functions, or Fourier methods, or renormalization. Physicists used each of these for decades or even centuries before their methods were rigorously proven correct. In each case, one could construct pathological examples in which the tool broke down, yet physicists in practice had a good sense for what kinds-of-things one could and could not do with the tools, based on rough informal arguments. And they worked! In every case, mathematicians eventually came along and set the tools on rigorous foundations, and the tools turned out to work in basically the cases a physicist would expect.
So there’s clearly some epistemic techniques here which aren’t captured by Mathematical Proof + The Scientific Method. Physicists were able to figure out correct techniques before the proofs were available. The Scientific Method played a role - physicists could check their results against real-world data - but that’s mostly just checking the answer. The hard part was to figure out which answer to check in the first place, and that involved informal mathematical arguments.
We don’t really have a legible Art of Informal Mathematical Arguments, the way we have a legible Art of Mathematical Proofs or Art of Scientific Method. Informal mathematics clearly played a key role historically in figuring out useful tools, and will likely continue to play a key role in the future, but we don’t have a step-by-step checklist to follow in order to use informal mathematical arguments (the way we do for The Scientific Method), or even a checklist to verify that we’ve applied informal mathematical arguments correctly (the way we can for Mathematical Proofs). If someone says that my informal mathematical argument is wrong, and I can’t either convert it to a formal proof or show some definitive experiment, then I don’t have a clear standard way to argue that it’s correct.
Yet there’s clearly something making informal mathematical arguments correlate quite highly with truth (if imperfectly), because they work well in practice!
The same applies to Fermi estimates. They work remarkably well in practice, yet there’s not a standard step-by-step checklist. There’s not some standard rules to check whether a Fermi estimate is correct. If someone disagrees with my Fermi estimate, there’s not a way for me to establish correctness other than putting in all the work to find more-rigorously-estimated numbers.
The same applies to gears-level models. Plenty of physicists (and engineers, and others in technical fields) have an intuitive sense that a gears-level model is useful and powerful in ways that a black-box model isn’t. But if someone comes along and says that their 50-variable linear regression gives more precise predictions than a model based on the internal structure of the system, and that we really don’t need those gears anyway, I expect most people would not have a strong explanation of why the gears matter. Such explanations do exist (see e.g. the Lucas Critique, or my own writing on the subject), but gears-level models are still in the early stages of becoming legible. As of today, we don’t even have a widely-accepted explanation/definition of what “gears-level” means! For most practitioners, it’s just a vague aesthetic sense, at most.
The really important point to notice here is not any one method, but that there seems to be a bunch of these. Enough that there’s probably more of them which we haven’t even given names to yet. And they seem to come disproportionately from physics. (Or from the kinds of applied mathematicians who are adjacent to physics, and not adjacent to people who write Definition-Theorem-Proof style textbooks and papers.)
So if we want to learn all these key illegible methods, use them ourselves, and maybe someday make them more legible, then physics (and physics-adjacent applied math) seems like the main subject to study.
One caveat to anyone following this advice: once you get past the basics, it is probably more important to study the work of physicists as opposed to physics per se. Even outside of physics itself, there are certain patterns of thought which will make it clear who the physicists are - for instance, I could guess that Uri Alon got his degree in physics, even though he’s known mainly for his introductory text on systems biology. It’s exactly those illegible epistemic tools which identify such people; once you have a little bit of a handle on the tools, you’ll hopefully be able to recognize other people using them. And by studying how those tools are used outside physics itself, we can get a wider view of their application.