I'd love to give recommendations on probability, but I learned it from a person, not a book, and I have yet to find a book that really fits the subject as I know it. The one I usually recommend is Grimmett and Stirzaker. It develops the algebra of probability well without depending on too much measure theory, has decent exercises, and provides a usable introduction to most of the techniques of random processes. I found Feller's exposition of basic probability less clear, though his book's a useful reference for the huge amount of material on specific distribution in it. Feller also naturally covers much less ground (probability and stochastic processes has developed a lot since he wrote that book). Kolmogorov's little book (mentioned elsewhere in the threads) is typical Kolmogorov: deliciously elegant if you know probability theory and like symbols. I would love to be able to recommend Radically Elementary Probability Theory by Nelson, and it's certainly worth a read as a supplement to Grimmett and Stirzaker, but I would hesitate to hand it to someone trying to understand the subject for the first time.
For statistics, I favor Kiefer's 'Introduction to Statistical Inference'. It begins with the decision theoretic foundations and builds from there, skipping or bypassing huge numbers of standard topics, and using a notation I can only describe as Baroque, but it is the best source of real understanding and intuiton I know of. Hogg and Craig's 'Introduction to Mathematical Statistics' is a pretty nice text as well, but less precisely pitched than Kiefer's (and it covers a lot more of the standard topics). Casella and Berger's 'Statistical Inference' and Lehmann's two books 'Point Estimation' and 'Hypothesis Testing' are the more typical graduate statistics texts, but are hard going compared to my other recommendations.
I'm going to disagree about Griffiths for electromagnetism, but admit that I don't have a really good alternative to offer. I found the second volume of Feynman clearer. Jackson is utterly opaque, a book length exercise in Green's functions methods in linear partial differential equations, and one without mathematical rigor. Schwinger's 'Classical Electrodynamics' is actually a remarkably useful text. I would probably recommend Purcell's 'Electricity and Magnetism', but it's out of print.
For thermodynamics, Hatsopoulos and Keenan's 'Principles of General Thermodynamics' is the best text I know. It's certainly better than any of the recommendations I received in my physics department. There are lots of beautiful texts -- Fermi's, Sommerfeld's, the opening couple chapters of volume 5 of Landau and Lifshitz, etc. -- but they all assume a developed conception in the student's mind of the nature of a thermodynamic system, while Hatsopoulos and Keenan spell it out in utter clarity. My only caveat about this book is that their exercises are given in Imperial units.
For statistical mechanics, I still think that Landau and Lifshitz volume 5 is the best text I know of. Sethna's 'Entropy, Order Parameters, and Complexity' is really neat, and touches on a lot more modern techniques, but has less real meat, less direct physics, than L&L. After that I think Reichl is probably my favorite, and he does set things up in a nice way, but not as nicely as Sethna.
Despite six years of wearing the big white suit in a tuberculosis laboratory, I am unaware of a microbiology textbook that should be read instead of burned.