All of Darren's Comments + Replies

Another key work here is Probability Theory: The Logic of Science by ET Jaynes. (you can download the entire book here). The early chapters are focused on deriving the probability calculus from logic. 

As you've noted, Bayes' Theorem is just a straight forward result of probability calculus. In that light, it is entirely uncontroversial. 

What people really seem to get excited about is Bayesianism, which is something more than just the application of Bayes' Theorem. 

To understand people's interest in Bayesianism, I think you then need to distinguish its use in two types of applications: how we use probabilities to deal with uncertainty when drawing inferences from data generated by scientific studies (i.e. statistical inference); and whether hum... (read more)

Yes, Bayesianism is more than one thing. (BEIMTOT) Theres a plausible version of Bayes, which isn't very exciting, the update rule. And an exciting version, Bayes as a complete system of epistemology, which isnt very plausible. In particular, it isnt able to answer questions like "what is evidence?" and 'where do hypotheses come from?" ... leaving most of the vexing questions you would want a complete system of epistemology to solve, unsolved. So you have all the ingredients for motte-and-bailey confusions -- two things that come in exciting but implausible and plausible but boring versions, and they're called by the same name.
You might find this reference useful: Bayesian Epistemology. Personal view: if you think you're capable of forming reasonable priors, you're "probably" a Bayesian.