Is there any conceivable way of telling whether we're in a simulation
Yes, with high probability at least. We create lots of simulations of observers who have the same subjective beliefs as we do. Not that I'm in favor of this, but you did use the word conceivable.
Three tips for reasoning about the real world, for people who might be "living in a simulation" or otherwise inhabiting a systematic illusion:
(1) If: every world in some ensemble of possible worlds exists, and you know enough about this ensemble; then you can reason about the properties of the real world, by applying standard probabilistic inference to the set of all entities, occurring in this multiverse, which are subjectively identical to you. For example, if half of your subjective duplicates across the multiverse inhabit worlds made of superstrings, then the probability is 50% that your real world is made of superstrings.
This is both a no-brainer and arguably useless advice, since you just may not be in a position to know the full nature of the multiverse, or to calculate the demography of simulations and false realities throughout it. However, it is worth understanding that if we could know enough, then answering these questions really would be a matter of deduction and calculation.
(2) In the absence of a justified, top-down way of answering questions about the real world, you can at least aim for a systematic approach by collecting and generalizing hypotheses. Nancy has a small list above ("dreams, daydreams,..."); this could be expanded.
There are lists of proposed answers to the Fermi paradox (1, 2), perhaps it's time to start work on assembling comparably comprehensive lists of simulation scenarios.
(3) One might also wish to discreetly make a list of ways that making a list of simulation hypotheses could induce feedback from the laws or the lords of the matrix. There are simulation scenarios in which you are not supposed to know, or in which trying to learn the nature of the simulation has prearranged consequences. These could be called "simulation-fights-back" scenarios.
Edited to add: I should think that being a simulation is an existential risk.
Just being is an existential risk. Counterfactuals are much safer.
If we're part of a simulation, how likely is it that whatever it's running on is using the same sort of atoms we've discovered?
Even if we right now had the ability to run really good simulations we will probably run a larger variety of simulations with different types of physics with different building blocks than the building blocks of our reality (a crude example - Conway's game of life does not simulate atoms) so I'd reason that its more likely that the real physics of the real world are not very similar to our physics. And even if they are similar, our physics are more probably just an approximation of the real thing.
This is also the reason why I don't think we are very likely to be in an ancestral simulation - the intelligence(s) in the real universe that has/have the capability of running such an elaborate simulation of our world are likely to have ran an enormously large numbers of simulations of universes not like its/their own and the number of those simulations is probably way way way higher then the number of ancestral-type simulations that it/they will have run.
If anything the likeliest type of simulations to be data-mined (as in to be ran in the largest number of the possible varieties) which are in accord with whats going on in our world would be simulations of universe with intelligent life that leads to a singularity. In fact even ancestor simulations suggest that we are in a simulation that reaches a singularity (how else are they running a large number of such complex simulations). My reasoning for this is that data-mining of such simulations is likely done by a sort of superintelligence and it is reasonable to argue that a superintelligence will be searching for/find useful data from a post-singularity world and not so much data of what some stupid apes do.
So onwards to practical questions. Is there any conceivable way of telling whether we're in a simulation and if so, learning something about its nature? Is it worth trying to get out of the Big Box?
Every test can be faked by the lords of the matrix in a way that always gives the results that we will get in the 'real' universe, so there might not be a way to tell if we live in a simulation. However the more it seems that we will be able to run complex simulations with life that experiences qualia the more likely it is that there are a large number of such simulations per real universe which makes it more likely that we live in a simulation. If the evidence gets good enough we might one day know that the chances are only 10^100 to 1 for us to be in the real world (a made up number just to illustrate my point).
This is also the reason why I don't think we are very likely to be in an ancestral simulation
I use stronger language. The chance that we are in an ancestor simulation is in the neighborhood of epsilon.
All the arguments which imply that we might be living in a simulation would imply that the universe which is simulating us is also a simulation. I assume there's got to be a ground universe somewhere in the chain.
An extreme Singularity doesn't seem necessary to get a lot of simulations.
Whether we can be sure we've gotten out may be a different problem than whether we can get out.
I assume there's got to be a ground universe somewhere in the chain.
I'm not saying you're wrong to think this is likely, but I don't think this is as necessary a condition as some people are taking it to be. So long as each simulation is simulated from somewhere, there's no reason why it can't be the case that every simulator is also simulated. I can think of no reason why the universe would be like this, but I can also think of no reason why it can't be that way.
The chance that we are in an ancestor simulation is in the neighborhood of epsilon.
I also put the chance for an ancestor simulation in that neighborhood, I just don't give a high probability that my best guesses on the topic are actually sound and that skews things a little. ( as in 'There might be reasons why a simulation-capable society would run more ancestral simulations than other kinds of simulations even if I haven't thought of those reasons yet')
All the arguments which imply that we might be living in a simulation would imply that the universe which is simulating us is also a simulation. I assume there's got to be a ground universe somewhere in the chain.
Yup, there must be a base universe but it is possible that the inhabitants of the base universe have also done their math and also reason that the chances are higher for them to be in a simulation (even though they are actually the only ones(not actually sure about the 'only' bit) that are not in a simulation).
An extreme Singularity doesn't seem necessary to get a lot of simulations.
Not an extreme one, no. Still, you need to have passed a certain level to be able to run those simulations and you can progress your civilization a lot more just from information that you gather from such simulations which in turn would logically lead to an 'extreme' singularity. (exponential growth and all).
I agree that full-universe ancestor simulations are unlikely. Limited simulations focused on interesting historical events and people seem much more likely (alongside more exotic simulations, of course). Your probability that you're a part of such a simulation depends on the interestingness of people and events proximal to you. Robin Hanson wrote a paper on how to behave in such a simulation.
FWIW, I don't have a high probability of being in a simulation like that; I'm not close enough to anyone or anything historically interesting to be simulated in full, conscious detail. But someone like Robin Hanson, EY, etc., must have a much higher probability.
How much gets left out of simulations is highly dependent on the cost of simulations and the interests of the viewers.
We might be in a naturalistic simulation where what we'd call boring people are as carefully simulated as anything else. I'm much more willing to bet that other galaxies are merely sketched in.
It seems reasonable that if we're in a simulation, then the universe we're simulated in has much more resources than we do. There would still be more and less elaborate simulations, but the odds of being in the range where how interesting you are matters strike me as too small to bother with. It's like trying to optimize winning the lottery when you're an average person, but with much less scope sensitivity. (Mathematicians occasionally find a flaw in a lottery.)
I've been playing with the idea of what do you do if you get out of the Big Box, and I don't think it's take over the world-- it's get your universe copied on to more different computers, possibly on to more different platforms. I'm not sure that people like us can run directly on their physics.
If we're part of a simulation, how likely is it that whatever it's running on is using the same sort of atoms we've discovered?
I think the answer is it's very unlikely. The closest resemblance I find plausible is that our atoms are simplified versions of the substrate atoms, and I wouldn't count on even that much.
I'm pretty sure that a simulation has to be smaller in some sense than the universe that's running it, which means that it has fewer things or simpler things (these might be equivalent because more simplicity means fewer sub-components in things) than the home universe.
You might do a meticulous job of simulating your matter in a simulation, but I suggest that you'd only bother in a small and/or specialized simulation, and even if you did, there's a reasonable chance that you don't have a full understanding of your own physics.
When I look at the range of human-created simulations (dreams, daydreams, fiction, games, art, scientific, political, and commercial simulations) and contemplate that we've probably only explored a small part of the possibilities for simulation, it seems vanishingly unlikely that we're in an ancestor simulation.
When I first came up with the question of the nature of our possible substrate, I didn't think there was a way to get a grip on it at all, but at least now I've got some clarity about the difficulties I think.
So onwards to practical questions. Is there any conceivable way of telling whether we're in a simulation and if so, learning something about its nature? Is it worth trying to get out of the Big Box?
Edited to add: I should think that being a simulation is an existential risk.