Easy Predictor Tests

byMrHen9y21st Jan 201066 comments


A fun game you can play on LessWrong is to stop just as you are about to click "comment" and make a prediction for how much karma your comment will receive within the next week. This will provide some quick feedback about how well your karma predictors are working. This exercise will let you know if something is broken. A simpler version is to pick from these three distinct outcomes: Positive karma, 0 karma, negative karma.

What other predictors are this easy to test? Likely candidates match one or more of the following criteria:

  • Something we do on a regular (probably daily) basis
  • An action that has a clear starting point
  • Produces quick, quantifiable feedback (e.g. karma, which is a basic number)
  • An action that is extremely malleable so we can take our feedback, make quick adjustments, and run through the whole process again
  • An ulterior goal other than merely testing our predictors so we don't get bored (e.g. commenting at LessWrong, which offers communication and learning as ulterior goals)
  • Something with a "sticky" history so we can get a good glimpse of our progress over time
A more difficult challenge is predicting karma on your top level posts. My predictors in this area tend to be way off the mark. For this post, my guess is between +4 and +20. Reasoning: I don't see how it could get over 20 unless it gets promoted and the concept surprises readers; 4 seems like a solid guess for "interesting, uncontroversial, but not groundbreaking."
Update: As of January 29, this post is at +10.