This might be a minor or a major nitpick, depending on your point of view: Laplace rule works only if the repeated trials are thought to be independent of one another. That is why you cannot use it to predict sunrise: even without accurate cosmological model, it's quite clear that the ball of fire rising up in the sky every morning is always the same object. But what prior you use after that information is another story...
That entirely depends on your cosmological model, and in all cosmological models I know, the sun is a definite and fixed object, so usually
The premise seems to be that there is no model, you're seeing the sun for the first time. Presumably there are also no starts, planets, moons in the sky, and no telescopes or other tools that would help you build a decent cosmological model.
In that situation you may still realize that there is one thing rotating around another and deduce that P(sunrise) = 1-P(apocalypse). Unless you happen to live in the Arctic, or your planet is rotating in some weird ways, or it's moving in a weird orbit, or etc.
My point is that estimating P(sunrise) is not trivial, the number can't just be pulled out of the air. I don't see anything better than Laplace rule, at least initially. You said it doesn't work, so I'm asking you, what does work?
A much, much easier think that still works is P(sunrise) = 1, which I expect is what ancient astronomers felt about.
Take a prior over all potential turing machines, take the number of all the agents simulated by those turing machines (times their respective turing machines prior probability) that have had the same sensory experience that you remember and that see a sunrise tomorrow, divide by the number of agents (multiplied by the prior for their respective turing machine) that have had the same sensory experience and do not see a sunrise. Done. Trivial.
I think this is a very valuable concept to keep fresh in the public consciousness.
However, I think it is in need of better editing; right now its formatting and organization make it, for me at least, less engaging. This is less of an issue because it's short; I imagine that a longer piece in the same style would suffer more reader attrition.
It might help to read over your piece and then try to distill it down to the essentials, repeatedly; it reads right now as if it is only a few steps removed from straight stream-of-consciousness. Or it might not; at this point I'm speculating wildly about the creative processes of someone I've never met, so take my implementation advice with a grain of salt.
Either way, I look forward to reading more of your insights.
My mindset was that this was a social site rather than a publishing depot so I didn't put the effort into editing. But I'm also new to the forums (although longterm reader of the extended LW universe) so I'm happy to be proven wrong if need be.
I'm amazed that no one critiqued the most glaring error- that the sun will not supernova but rather turn into a red giant and then a white dwarf and that you would be not be able to notice it not rising because you'd be dead. X )
I think you're making a mistake here. The connection between P(the sun will rise tomorrow/there will be a source of light in the sky tomorrow) and P(the sun will continue to exist forever/this will continue forever) is not trivial, and you are confusing high confidence in P(the sun will rise tomorrow) with a mistaken confidence in the additional claim "and this will continue to be true forever".
I think the correct way to do Bayesian updating on this situation is to consider these two hypotheses separately. P(the sun will rise tomorrow) will behave according to a Laplace rule in your world. But P(the sun will consist forever) should have a very low initial prior as anything existing forever is an extraordinary claim, and observing an additional sunrise is only weak evidence in favour of it being true. Conversely the decay of other objects around you (a fire running out, for example) is weak evidence against this claim, if only by analogy.
In the spirit of only trying to explain that which is actually true I think it's also worth noting that the sun visibly changes quite a lot before extinguishing, so an ideal Bayesian would correctly deduce that something extraordinary is happening. In the presence of a sun that will soon extinguish our Bayesian agent will remark that the inductive argument 'the sun rose every morning of my life, therefore it will exist forever' doesn't properly explain the changes that are visible, and the hypothesis will take a corresponding hit in probability.
My intent was less to imply p(the sun will rise forever), but rather that the p(the sun will rise tommorrow) will eventually be very wrong. Of course as you mention the sun would start to look weird which should trigger uncertainty, but that is more the particulars of the example, than the spirit of the thought experiment behind it imo.
So I've been thinking about Bayesian probability and paradigm shifts. One of the early examples that Price published after discovering Bayes' theorem (after Bayes died) was of someone who, upon awakening for the first time with no other information on cosmology, if they knew Bayes theorem, could then update their probability that the sun would rise again the next day, each day they saw it rise again. So with time, as they see the sun rise more and more times, they become more and more 'certain' that it will rise again the next day (ie their priors become higher).
However, not having any knowledge of the universe or physics, they are unaware that there is a near certainty that this sun will someday supernova and no longer rise again. If they made thousands of generations of sun tracking bayesians, every day they would see the sun rise and update their probability, and become more certain that it would rise again. By the time it didn't rise, they would be wildly certain that it would rise again. So the more certain they became, actually the more WRONG they became. That sun was always almost certainly doomed at the same 99.999....% level the whole time (maybe not to each given new day, but eventually) and they just didn't have access to good enough priors to recognize this.
So as a result of bad priors, they are maybe increasing their accuracy relative to any given day (the sun only dies on 1 in a billion days) but decreasing their accuracy of it's eventual transformation into a black hole or some such phenomena which will likely kill the shit out of them.
I think this kind of misinformed search for accuracy is very symbolic of a bayesian look at paradigm shifts (even as it could also b used as a limited critique of bayesian statistics). Once they get access to just the knowledge that other stars exist, it opens up a huge range of other variables they didn't know about in the calculation of their priors. So while we're chugging along in our search for accuracy, we may be building relative accuracy, while building absolute an error until our paradigm catches up with a new and deeper layer of information.