How would you write a better "Probability theory, the logic of science"?

Brainstorming a bit:

  • accounting for the corrections and rederivations of Cox' theorem

  • more elementary and intermediate exercises

  • regrouping and expanding the sections on methods "from problem formulation to prior": uniform, Laplace, group invariance, maxent and its evolutions (MLM and minxent), Solomonoff

  • regroup and reduce all the "orthodox statistics is shit" sections

  • a chapter about anthropics

  • a chapter about Bayesian network and causality, that flows into...

  • an introduction to machine learning

My perspective on anthropics is somewhat different than many, but I think that in a probability theory textbook, anthropics should only be treated as a special case of assigning probabilities to events generated by causal systems. Which requires some familiarity with causal graphs. It might be worth thinking about organizing material like that into a second book, which can have causality in an early chapter.

I would include Savage's theorem, which is really pretty interesting. A bit more theorem-proving in general, really.

Solomonoff induction is a bit compl... (read more)

1Daniel_Burfoot4yAny book on statistics needs to have a section about regularization/complexity control and the relationship to generalization. This is an enormous lacuna in standalone Bayesian philosophy. I now see that most of Jaynes' effort in the book is an attempt to repair the essential problem with Bayesian statistics, which is the subjectivity of the prior. In particular, Jaynes believed that the MaxEnt idea provided a way to derive a prior directly from the problem formulation. I believe he failed in his effort. The prior is intrinsically subjective and there is no way to get around this in the traditional small-N data regime of statistics. Two observers, looking at the same small data set, can justifiably come to very different conclusions. Objectivity is only re-achieved when the size of the data set becomes large, so that Solomonoff-style reasoning can be used.

Open thread, Jul. 25 - Jul. 31, 2016

by MrMind 1 min read25th Jul 2016133 comments


If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "