I think there is an objectively right direction to shift, given particular priors. Your anti-regularity observer seems to be making a mistake by becoming more confident if he actually sees heads come up next.
Also, I edited my post above to fix a notational error.
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.