This post is motivated by Joe Carlsmith's post in which he argued some ways of understanding SIA undersells its appeal. He purposed a better way of stating SIA. I disagree with some of the assessments.
Before I start, it should be pointed out that all different ways of stating SIA are computationally equivalent. So it is unlikely that one interpretation is "correct" while the other is "wrong". But it may help us in evaluating SIA's validity.
First is the ordinary SIA statement. If you check on Wikipedia or the Lesswrong Wiki you can find SIA as:
All other things equal, an observer should reason as if they are randomly selected from the set of all possible observers.
This is actually slightly different from Nick Bostrom's original statement of SIA in his book "Anthropic Principle". But it is logically the same. And since then it has been most widely used including Bostrom himself in some interviews (if my memory is correct). Furthermore, the criticism presented by Carlsmith applies to this formulation. So I will use this statement.
Carlsmith presented SIA as:
You’re more likely to exist in worlds with more people in your epistemic situation. Or update the prior of objective worlds in proportion to n (the number of people in your epistemic situation).
I intend to compare the two and argue why the ordinary statement in my opinion is still better.
Carlsmith argued the ordinary statement uses the notion of reference-class. Yet its definition is arbitrary. E.g. who are the "observers"? How do you define them? Even though the choice of reference-class will not affect the answer in many anthropic problems, due to its effect cancels out, this "inflate-and-claw-back" method is undesirable. The appeal of SIA is not "you can use whatever reference class you like" but you don't have to think in terms of any made-up reference class at all. You only need to consider yourself a member of "people in your epistemic situation".
But "people in your epistemic situation" is not a clear-cut definition either. An epistemic situation is a subjective state, and the only subjective experience you have is that of your own. So what qualifies “people in your epistemic situation” has to be a judgment call, or in another word, made up. Do brains in jars that are being fed with similar neural signals count? as Adam Elga discussed in "Defending earth with self-locating probability". What about computer programs like Bostrom's Simulation Argument? Depending on your judgment of what “people in your epistemic situation” include, the answer to those problems would be drastically different. You are either certain you are a physical person or be quite confident that you are just a brain/program.
These judgment calls don't matter only in problems where their effect cancels out. In this sense "people in my epistemic situation" is just another choice of reference class. Obviously a valid choice, because you can use whatever reference class you define. But it does not solve the arbitrariness in SIA's formulation.
Another problem with the new statement is it only discusses your chance of existence. Yet one important function of SIA is regarding how to calculate self-locating probabilities. For example, in Sleeping Beauty Problems, after waking up, what is the probability that today is Monday?
The ordinary statement got it covered. Consider "today" a random sample of two possible days, eliminating Tuesday/Heads after waking up, therefore the probability of today being Monday is 2/3.
The new statement says nothing about it. It will update the probability of Heads to 1/3 due to more "people-moment in this epistemic situation" existing if Tails. Yet what is the probability that today is Monday? The value of 2/3 requires a principle of indifference that is not reflected in the statement. In part IV of the series, Carlsmith also thinks the principle of indifference is central to SIA. So the complete new statement should be something like:
You’re more likely to exist in worlds where more "people in your epistemic situation" exists AND you are equally likely to be any one of the existing "people in your epistemic situation".
Once put this way, the new statement loses its parsimonious appeal. Furthermore, the second half of the statement looks very similar to SSA (the difference is the reference class used). I would argue this is closer to Bostrom's original SIA+SSA formulation. Making SIA look more like an additional assumption on top of SSA. Whereas the ordinary statement is simpler, clearly demonstrates its distinctiveness from SSA, i.e. sampling from potentially existing observers vs actually existing observers, thus is easier to understand.
As said earlier, all these statements are computationally equivalent. So I wouldn't say one is more "correct" than another. The very fact that stating the same concepts differently would make the argument seems more plausible is not something to get comfortable with. It should make us cautious of SIA (and SSA as well)'s validity.