My experience forecasting has led me to realize that a key mistake that is often made is updating on new data too quickly. This seems backwards, but I think that often the biggest reason that people both under- and over-react to evidence is that they don't consider the evidence clearly enough, and immediately start revising their model to account for the evidence, instead of actually thinking before updating.
Let's deconstruct how rapid updating is misleading with a simple notional example. Someone shows up with a coin and claims that she is psychic and can predict coinflips. You are skeptical, and challenge her to do so. She immediately takes out a coin and correctly predict heads 3 times in a row. You can update a few ways:
- Conclude that the claim is now much more likely than before, and you give some credence to the idea that she is psychic
- Conclude that she was lucky, and 7:1 odds is not very strong evidence, so you continue to hold on to your prior strongly
- Conclude that she is cheating using a coin which has heads on both sides
Notice that these possibilities which spring to mind are completely different cognitive strategies. Either you update to believe her more, you decide the evidence is insufficient, or you immediately seize on a new hypothesis that explains the evidence.
But these are all mentally lazy strategies. If you thought about the situation for longer, you could easily generate a half dozen additional hypotheses. Perhaps the she has trained herself to flip coins a certain way. Perhaps she simply lied and said the coin landed heads each time, and didn't really give you time to see it well when it didn't. Perhaps she is substituting the coin as it lands. Perhaps, perhaps, perhaps.
My advice, per the title, is to slow down. You might decide to be a good Bayesian, and preserve multiple hypotheses, updating marginally - but doing this means that you assume the correct hypothesis is in your prior set. There are a million hypotheses that can explain a given set of events, and the most cognitively available ones are those that allow you to be lazy.
Don't worry, take your time. If the issue matters enough to bother trying to update your model, taking five minutes to reflect is better than jumping the gun. And if you don't need to make any decisions, at least file everything away and decide that it's unclear instead of quickly responding with an overconfident "Aha, now I understand!" or worse, a "Eureka, I've solved it!"
Bayesian thinking gives you answers no faster than a rational accumulation of evidence can possibly allow, given the uncertainties that exist. Slow down. Marginally rationally updating doesn't give you confident answers quickly. It can't.
Trying to update faster isn't going to get you better answers now, it will get you worse answers more quickly.
Updating isn't a sprint to the answer, or even a race - it's a long-duration hike towards a still-unclear goal. If you imprudently start sprinting early because you think you see the goal, you're just going to hurt yourself, or get lost and never make it to the right destination. Take your time.