There are probabilities, and there are probabilities about probabilities. How do these get updated? I've had the same discussion several times, and have tried to describe this, but it is hard without going into the math. The formal model is clear, but I have found that the practical implications are hard to describe concretely. I just ran into a great concrete example, however, and I wanted to work through the logic of how I'm updating as a way to show what should happen.

The example I'm using is my expectations about the 2020 election, how accurate various models are${}^1$, and how important the inputs are. This type of problem is fairly common - I have both an object level prediction about the winner, and a prediction about / model of how accurate different sources of information will be.

So, what do I do when information comes in that seems surprising? Two things; I update in the direction the information indicates, and I update against the reliability of the data. The second may seem counter-intuitive, but the example makes it clearer.

The economy is doing well - recent news is that it's better than expected. Presidents with great economies tend to get re-elected. Trump is also unpopular. Unpopular presidents tend not to get re-elected${}^2$. How do we balance these two, and how do they interact? My model of whether he will win is fairly uncertain, and my model of the sources of data is also uncertain. They are also related in complex ways${}^3$. For instance, if Trump's popularity plummets because, for instance, the impeachment inquiries find something shocking and horrible even to his base, I expect that GDP matters far less for his reelection chances. Other data sources also constrain how far I will update${}^4$ - no level of GDP growth alone will make me say he's certain${}^5$ to win.

So I updated towards Trump's reelection based on the economic data, but my underlying model is telling me that it is decreasingly relevant. That means I'm very slightly down-weighting the importance of economic factors compared to approval rating, since he's seemingly not getting credit for the growth (or the growth isn't helping most voters.) The net impact is that I have updated slightly towards Trump's reelection.

1) For long term forecasts of presidential elections, forecasts based on fundamentals do just OK. But forecasts based on polls do poorly far in advance of the election as well. (Special elections seem to point to a huge shift towards the democrats, despite fundamentals.) More complete models take some of each type of information - but how to combine them is tricky. Some models do it poorly, others do it well.

2) I also have expectations about the future inputs to the models. Most presidents have fluctuating approval ratings, so long-term forecasts do poorly. For Trump, his split of approval/disapproval has been remarkably steady, so unless his approval significantly shifts from the current low-40s, or he runs against an incredibly unpopular democrat (which is possible, but seems pretty unlikely,) models that consider this point towards him being unlikely to win. It still may be volatile. For example, the impeachment could solidify his base, or could reduce his popularity further.

3) This is tricky to describe, but for understanding the overall behavior, a useful strategy is to consider the limit - what happens if the economy is amazing, but everyone hates the president? I'd assume he doesn't get reelected. Similarly, if everyone loves the president, but the economy is in a deep recession, (for which he's seemingly not being blamed) he probably gets reelected.

4) Special elections are favoring Democrats, voter turnout among liberals is expected to be very high because of polarization, etc.

5) By which I mean highly confident - certainty is impossible. It would take a confluence of events to make my highly confident. Even with such a confluence of events, however, it is far in advance, so I'm not willing to put odds above ~90% / below ~10% because I think there are fundamentally hard questions about the future that impact the probability. (We don't know who the democratic nominee is, for instance.)