Have you ever desired external tools for the deliberate shaping of your mind?
If so, I’m pleased to introduce Calibrate an open source Android app for probabilistic forecasting calibration.
Do you wish that when you said something had a 75% chance, it really did happen about 75% of the time? That’s what calibration means, and Calibrate helps you get there.
The App works by you first creating your predictions, about anything you like: work, play, daily life, world events, habits etc.
Then once you have made a few predictions you can track them either in list form:
Or in 3 types of graph, here is the Accuracy graph, showing your calibration. Predictions are grouped by in adjustable probability bins for ease of visualization.
The Frequency graph shows how you assign different probabilities, helping reveal blind spots.
Finally the Log Loss graph shows your Log Loss (or Brier score toggled via filter) for individual predictions and as a rolling average, allowing you to see if you are improving.
Quality of life features include editing, filters, tags, light and dark themes, external predictions backup and restoration.
I was prompted to make the app after finding my mind generating probabilities on events more and more, reflexively, but frustratingly not getting any feedback!
Anecdotally when I put numbers on things my mind feels less sloppy, like I'm holding myself to a higher standard of thought. It’s motivating to know that any sloppy overconfidence will show up as a bump in your Log Loss curve.
A benefit I didn't foresee is as a motivation tool, for example if you give a probability you will begin work on some bothersome chore that day, assigning a low probability feels like a defeat, so I assign a high one, then feel compelled to actually follow through.
It also serves as a convenient private history of your beliefs.
I plan to maintain and extend functionality, so if anyone finds use in it, please let me know and feel free to drop bugs or feature suggestions in the comments at any time.
A pocket tool for Bayesian calibration.
Have you ever desired external tools for the deliberate shaping of your mind?
If so, I’m pleased to introduce Calibrate an open source Android app for probabilistic forecasting calibration.
Do you wish that when you said something had a 75% chance, it really did happen about 75% of the time?
That’s what calibration means, and Calibrate helps you get there.
The App works by you first creating your predictions, about anything you like: work, play, daily life, world events, habits etc.
Then once you have made a few predictions you can track them either in list form:
Or in 3 types of graph, here is the Accuracy graph, showing your calibration. Predictions are grouped by in adjustable probability bins for ease of visualization.
The Frequency graph shows how you assign different probabilities, helping reveal blind spots.
Finally the Log Loss graph shows your Log Loss (or Brier score toggled via filter) for individual predictions and as a rolling average, allowing you to see if you are improving.
Quality of life features include editing, filters, tags, light and dark themes, external predictions backup and restoration.
I was prompted to make the app after finding my mind generating probabilities on events more and more, reflexively, but frustratingly not getting any feedback!
Anecdotally when I put numbers on things my mind feels less sloppy, like I'm holding myself to a higher standard of thought. It’s motivating to know that any sloppy overconfidence will show up as a bump in your Log Loss curve.
A benefit I didn't foresee is as a motivation tool, for example if you give a probability you will begin work on some bothersome chore that day, assigning a low probability feels like a defeat, so I assign a high one, then feel compelled to actually follow through.
It also serves as a convenient private history of your beliefs.
It’s Android only, and not on the Play Store, you can download the .apk install file from GitHub to install (or of course build it yourself if you prefer).
I plan to maintain and extend functionality, so if anyone finds use in it, please let me know and feel free to drop bugs or feature suggestions in the comments at any time.