Even in markets where a positive resolution doesn't literally result in the death of most people alive, this can be a concern.
I've been tempted in the past to place money on nuclear war markets, but the fact that most prediction markets are pegged to USD makes me hesitant.
As an Australian, if a nuclear war breaks out, I would predict a much higher likelihood for my local government to keep meaningfully functioning than for nearly any other government in the world. Therefore, the value of Yes on a nuclear war market for me is meaningfully higher than that of most of the rest of the userbase.
But since the market resolution is pegged to USD, I can't meaningfully express that. If I could get some financial product that converted USD expressed smart contracts to AUD expressed ones (maybe backed by FX futures?) that would help, but for obvious reasons nobody is supplying these.
TL;DR: Whether Ćirković et al.'s Anthropic Shadow is philosophically legitimate or not, it contains a totally relevant intuition for profiting off prediction markets: if you're dead when the taxman comes, it doesn't matter how much you owe. As such, there's either an opportunity to make money by betting against x-risk-correlated events, or (once everyone's figured this out) prediction markets will systematically underestimate x-risk-correlated outcomes. In the post, I summarize a bit of the general debate about the Anthropic Shadow; then I present a toy model for the effect described, and a generalized version. I also list some candidate x-risk-correlated markets that could be vulnerable to the Anthropic Bias. Finally, I bring up and address some limitations I could think of—probably there are lots more, and I'd love to hear what they are!
In 2010, a group of astronomers and philosophers (including Nick Bostrom) published a paper arguing that we might be underestimating some kinds of existential risk, because we weren't taking into consideration the fact that we're all alive.
For example, if a very-deadly asteroid strike had ever occurred, it would probably have killed everyone—leaving no one to record it or study the frequency of deadly asteroid strikes. As a result, our own frequency-estimates of asteroid strikes may well be systematically biased, because we can only see asteroid-strike-histories compatible with continued existence.
In other words, all the counterfactual realities where catastrophic events have played out cast an "Anthropic Shadow" over our risk estimates: no observers exist within them, so we can never really see a fully-truly-representative sample.
Implicit in the paper is an acceptance of Bostrom's Self-Sampling Assumption (SSA), which holds that you should reason as if your existence was plucked randomly from the set of all existing observers, past, present, and future. The SSA is also behind Bostrom's earlier Doomsday Argument, which says that it's really surprising how early in the universe we all exist—how it's strange that we're here now, when there are so few people around, and how that implies that we probably don't have a very vast future full of observers ahead of us.
There's been a ton of debate in these parts on whether the Doomsday Argument is masturbatory nonsense, whether the Anthropic Shadow is masturbatory nonsense, whether the entire field of anthropics is masturbatory nonsense, and so on...
But one of the most interesting recent outputs of this discussion was Anatoly Karlin's Anthropic Shadow Geopolitics. In it, he argued for the existence of a sort of Reversed Anthropic Shadow:
Worlds that “make it” remember and/or simulate their history, whereas those that die, do not. This loads on what is known as the Strong Self-Sampling Assumption (SSSA), which relates to the idea that our current observer-moment - e.g., you reading this now - was plucked out of all possible observer-moments (past, present, and future) in their reference class.
(Quick note: I think that what Karlin calls the Strong Self-Sampling Assumption actually looks a lot more like the Self-Indication Assumption: SSSA can't deal with possible observer-moments, it only applies to actual, existent ones. SIA says things like "a world where a bajillion people come into existence is a bajillion times more likely than a world where one person exists," so it lets us update toward our reality eventually coming to contain many bajillions of lives; SSSA can't say anything about non-existent observers, which is why it pushes in the Doomsday direction Bostrom initially described: all it knows is that we exist among 8 billionish people's worth of moments (100 billionish including past lives), which suggests that not so many more are likely to exist. So when I quote Karlin saying "SSSA," I'm going to replace it with "[SIA].")
Karlin writes:
The validity of the [SIA] would be optimistic for some conscious observers surviving past AGI. But there is a catch. If you also believe that AGI is inherently dangerous; that AI timelines are short; and that alignment doesn’t happen by default and is inherently hard… then you must seriously entertain the possibility of a catastrophic but non-extinction level event between now and the early 2030s that rolls back AGI timelines by years or decades.
In other words, if SIA predicts an incredibly vast future, and AGI represents a real threat to that future, the fact that we're getting so close to superintelligence implies that something big and weird should happen soon to stop us.
I think Anatoly Karlin is probably wrong. I think it's unlikely that SIA suggests we have a vast future ahead of us no matter what,[1] and even if you think it does, you should probably mostly ignore that conclusion and continue working on AI safety in case you're wrong.[2]
But his article did inspire a related idea: namely, the fact that there really is an Anthropic Shadow hanging over us as individuals—one which has the strange effect of cancelling all bets in the case of an extinction event.
This is a pretty obvious conclusion: it's probably a large part of why Polymarket and Kalshi don't have any markets asking "Will humanity go extinct by [X]?" Because everyone can simply buy "No" and turn a guaranteed profit—if they turn out to be wrong, then no one will be around to tell them so.
But the big prediction marketplaces do have lots of questions sort-of-tangentially-related to extinction risk. Markets about the big, weird, and catastrophic things that Karlin tells us we should expect, conditional on extinction not happening.
To me, this suggests that either there's a bit of money to be made betting on anti-extinction events—or, if investors have already figured this out, a fairly strong bias toward these events' occurrence.
Let me illustrate with a toy model:
Imagine I'm going to roll a d6, and I ask how much you'd be willing to pay for a contract that pays out $1 if it comes up 4, 5, or 6. Probably you would say $0.50.
Ah, but there's a catch: You know that if the die came up 1, I would consider that, frankly, totally catastrophic. I would be so distraught that I'd cancel our contract and give you all your money back on the spot. How much should you be willing to pay for the contract now?
The situation is thus: 50% of the time, you'll get a payout of $1; the other 50% of the time, you'll have a 2/3 chance of getting nothing, and a 1/3 chance of getting your money back and breaking even. So if we call the price of the contract p, we should expect a return:
Which means you should be willing to pay as much as for a contract. If no one else in the market knows about my emotional reaction to rolling a 1, you can buy all their contracts off them for $0.51 and expect to make $0.09 a pop! Alternatively, if the information spreads, then my market, which is asking whether a d6 will come up 4, 5, or 6, will end up priced at $0.60. Which, you know, is wrong!
Of course, in reality, some existential risk exists no matter how the event turns out. So let's revise the situation a bit:
I'm a real klutz, so you insist on adding a line in our agreement that says, "If either party falls down a well before this contract is executed, it'll be totally cancelled, and everyone gets their money back." And let's say you have good reason to think there's a 10% chance I'll fall down a well before I'm able to roll the die.
Then we have a prior probability of bet-cancellation equal to 10%, and if the die comes up 1, it'll rise to 100%. If X represents the bet being cancelled and D the outcome of the die roll, we have:
So the expected return on a contract priced at p should be:
And so you should still be willing to pay up to for each contract.[3]
Let's try to generalize this principle: say you're betting on some event E that you think will reduce the odds of extinction X.[4] That is to say, . Ordinarily, you would pay a price up to for this contract—but when we update for the anthropic bias, we have...
Look out, there's math in here!
Astute observers will realize: that's Bayes' Theorem! Meaning the new breakeven price is simply equal to —in other words, what you would pay if you knew for a fact that the world wouldn't end / your contract wouldn't be cancelled. That seems like a pretty intuitive rephrasing of our initial question, "how should I adjust my bet for the possibility I won't be around to pay for it?" so I think this shouldn't seem like too alien a result.
Anyway, for our purposes, the more useful equation is...
So if you think there's a 10% chance of extinction right now, but that a world in which the next James Bond is British is probably one where everyone's a bit more chill—and your P(Doom|British Bond) is just 5%—then you should value each "Yes" contract at around times its normal price. (As of writing, that means buying at $0.71 rather than $0.67.)
Alternatively, if you think the market's already updated based on the British Bond–peacemaking effect, then you should think the true probability that a Brit takes the role is just of what Polymarket suggests—so the price should really be only $0.63.
[Notice: To be totally overly clear, none of this is investment advice. I'm just some online dumbass who's in no way qualified to give that sort of thing.]
The Anthropic Bias will only be significant if two things are true:
Motivation for (1) can be found at AI-2027.com and IfAnyoneBuildsIt.com—for (2), I suggest we look to Anatoly Karlin.
One candidate for his big, weird, catastrophic, Doom-averting thing is a war between the US and China over Taiwan.[5] This theory, to my eye, has a lot going for it! The Taiwan Semiconductor Manufacturing Company (TSMC) is responsible for the production of around 90% of the world's most advanced chips, and they house around 90% of their manufacturing capacity in Taiwan. In the case of a Chinese invasion, most of their operations would cease at least temporarily—in fact, it's likely that the Taiwanese would sabotage their own fabs rather than cede control to the CCP.
At the very least, this would retard progress toward AGI significantly—and that probably creates enough of an Anthropic Bias in favor of a "Yes" buy on 2025-expiring markets like this one.[6]
Other possibly-Anthropically-biased scenarios include (many of these are inspired by Karlin, though some I'm just making up on the fly):
A reduction in skilled immigration, either due to shocks and embargoes from a US-China war, or simple Trumpian-populist backlash. Silicon Valley, and the AI labs especially, have only been so successful and innovative because of easy access to huge amounts of talented human capital from India and China—if the US starts deporting foreign students and workers en masse, then it'll become much harder to leverage SV's vast economic capital to create AGI.[7]
I don't see any open markets that are very explicitly about high-skilled immigration, but they pop up from time to time, and even related-ish bellwether markets are probably exposed to some Anthropic Bias.
Here's one straight from Karlin: bioterrorism (and/or a good ol' fashioned natural pandemic). If lots of people die, then maybe we lower our odds of everyone dying! Unfortunately, the math gets fuzzier here: a deadly enough plague, say one that kills 50% of the global population (but also reduces extinction-odds), actually comes out pretty terribly when we're doing an Anthropic Update—a "Yes" resolution will only pay out in the 50% of worlds where you survive, whereas a "No" leaves you with normal P(¬Doom) survival odds. Fortunately, this means you can simply Bias in the other direction: against super-deadly pandemics![8]
I could really only find one related market: "New pandemic in 2025?" (depending on how deadly you think it'll be, the Anthropic Bias might point in either direction—though I think probably toward "Yes"). There's also "Bird flu pandemic before August 2025?," but it's sitting at 2%; no way the Anthropic Bias is strong enough to make any moves worthwhile there.
We could also see a general implosion of the tech world, which would probably significantly extend AGI timelines. Erik Hoel wrote a bit about this recently:
"Everyone knows Silicon Valley is poised (or at least it seems poised) on the verge of its greatest achievement in the form of Artificial General Intelligence. AI companies are regularly blitzed into the billions now. But you don’t need prophecies to predict some financial bubble popping or predict that the bar of AGI may be further away than it appears. You do need prophecies to talk about things more ineffable. About mythologies. About hero’s journeys. About villainous origins.
"For in the past few years, but especially this year, there is a sense that the mythology of the Valley has become self-cannibalizing, a caricature of itself. Or perhaps it’s best said as: it’s becoming a caricature of what others once criticized it for."
If the Valley becomes overly corporate, authoritarian, and evil, it could ultimately have a chilling effect on AI progress, as all its smart, techy counter-culturalists flee. Are there markets for this scenario? Kind of: "Magnificent 7 shrinks below 25% of S&P 500 in 2025?" seems pretty good; its "Yes" outcome might be Anthropically-favored.
What about a general implosion of the civilized world? Peter Turchin is a historian (ish) who's predicted impending (ish) End Times for the US, and really Western civilization in general. He's probably a crackpot, but of the coolest, most-data-driven variety—when he says "End Times" he means less Book of Revelations and more disintegration of all the important institutions and massive restructuring of who has power in society, as predicted by all these weird semi-mathematical models of social boom-bust behavior over millennia.
Probably that would be the sort of thing that makes AGI harder to build. (It also makes money less likely to be good for anything—but all the markets are crypto-based, so... should be fine?) Here are some questions I think might be exposed to an End Timesy Anthropic Bias: "Nuclear weapon detonation in 2025?," "US defaults on debt in 2025?", and "Will Trump withdraw the U.S. from NATO in 2025?."
Throughout this post, I've mostly considered "extinction" to be equivalent to "getting your money back." Does that actually make any sense at all?
No, it does not: when you place a bet, you lose access to that money until the market resolves. My model seems to imply that the world where you simply die before resolution is preferable to the world where you keep living, but lose the bet.
Luckily, there's an easy fix: if the Anthropic Bias is strong enough and your credit is good enough, you can profitably take out a small loan with which to place your bet. The cost is thereby transferred forward in time, cast beyond the Anthropic Penumbra.[11]
A bigger problem has to do with when we're expected to go extinct, exactly: Imagine there were a market asking, "Will the US President press his irreversible launch-all-nukes-in-exactly-three-weeks-button by the end of 2025?" Further imagine that this market was trading at $0.50, and that pressing the button would result in only a 10% chance of everyone on Earth being killed (the button was built by government contractors).
That feels like the sort of market you should be able to Anthropically update against:[12]
But you shouldn't go and sell a bunch of $0.45 shares! The market, presumably, will resolve well before its Anthropic effect is felt, and you'll be on the hook for betting 55¢ on a $1 coin-flip whether the world ends or not.
This is an effect you've really gotta watch out for: earlier, I mentioned that the anthropics favor betting "Yes" on "U.S. enacts AI safety bill in 2025?." I do think this is true, but for a much sneakier reason than "the AI safety bill will make AI safer, and extinction less likely, after the fact"—instead, it's based on the idea that passing an AI safety bill is probably indicative of a good, cautious approach to governing AI. It suggests that politicians and influential people have been doing lots of other good things in an AI-safety-direction that can reduce x-risk even before the bill passes.
Of course, it could be the case that an AI safety bill is actually anticorrelated with a slow takeoff: maybe worlds where we start racing off to dangerous AI are more likely to inspire an AI safety bill out of fear. In that case, we should probably update in the other direction: if we assume that, among the modalities where the government tries to pass an AI bill, many involve AI wiping us out first, then the contrapositive is true too: worlds where AI doesn't wipe us out are more likely to be worlds without an AI bill in the works—and so betting "No" is favored.
An Anthropically-minded bettor will have to decide which of these stories is more convincing before he can claim any sort of special knowledge over the rest of the market. To do that, he'll have to quantify all the various conditional P(Doom)s—which, of course, is very difficult, and brings me to the biggest problem with all this: namely,
Many of the longest-term markets on sites like Polymarket and Kalshi have expiration dates within a year.[13] This significantly hamstrings the power of the Anthropic Bias: your P(Doom) for the next year is probably not so large, and so you'll mostly find yourself dividing by something close to 0.9. See footnote 6 for a demonstration of how this leads to extremely lame results.
Worse yet, it can be really hard to precisely quantify the effect of various events on extinction risk. Is an AI safety bill a good sign or a bad one? Does war with China increase extinction risk on net (from nuclear war), or decrease it (from AGI)? These questions aren't so easy to answer, and your confidence interval will probably end up extending in each direction.
So if you want to beat the crowd, I think you're probably better off looking for alpha elsewhere. Get a job in the DoD, maybe, because the Anthropic Bias simply isn't big enough to bet on... yet.
As we get closer to AGI, P(Doom) will probably rise to a more significant level—maybe we'll find ourselves dividing by 0.7 or 0.6 for markets that resolve in 2027 (or 2028). And it'll also become clearer how AGI seems to be approaching, and which global events might prevent or delay its emergence. The margins will expand, the confidence intervals will shrink—and all of a sudden, the Anthropic Bias will rear its ugly (beautiful) head.
So, fine, maybe it's not such a huge deal right now—but do check back in a couple years...
To be clear, I do think the SIA is a good and legitimate view of anthropics. The problem is that I suspect it implies a vast multiverse full of experiencers, not necessarily a vast human future full of humans. Maybe the vast multiverse involves lots of souls floating around in heaven, maybe it's got to do with all the cockroaches that survive the nuclear holocaust and then reproduce forever, maybe it's about all the intelligent alien life out there, or the superintelligent dolphins that'll make it out in time and keep experiencing things, or AI could become conscious and build out a ton of slightly-unique observers, ...
Point is, we have little reason to think that humanity's extinction would spell the end of experience itself, and so I'm skeptical of SIA-based strong-anti-Doomerism.
It's also not clear that you would even want to stop working on AI safety if you bought completely into Karlin's idea: a world where mundane AI safety advances align AI is much better than one where e.g., nuclear holocaust destroys it.
This footnote will only make absolute sense once you've see the generalization, but for completeness' (the nerds') sake, here's why it comes out to the same price:
In the first case, our baseline and our , so we have .
In the second case, everything gets a little weird, but it all does cancel out. I'm going to write the equations in an appropriately weird, but unfortunately disgusting, way to demonstrate. Take to be the well-falling probability, and and to represent their values in the original case. and will represent their new ones. We have:
We can factor out of both the numerator and denominator and cancel:
So it doesn't matter how likely I am to fall down the well: as long as the bet will be cancelled on a roll of 1, you should pay $0.60 for a contract.
It's important to note this is "odds of extinction by the market expiration date" not in general! If the world doesn't end by the time you've got to pay, then none of this works!
(More discussion of this in the section titled "The Logistics Are Sort of Weird.")
By the way, if you're at all still tempted by Karlin's strong-anti-doomsday-anthropic-argument, the overreliance of global AI supply chains on Taiwanese semiconductors is fairly good evidence. It's extremely weird that AI progress is mostly dependent on one company which is mostly based on a rural, hilly, up-until-recently-very-underdeveloped little island that also happens to be one of the world's most dangerous political flashpoints.
I strongly recommend this review of TSMC-founder Morris Chang's autobiography: besides telling a fascinating story chock-full of good life lessons, it builds an excellent intuition for how ridiculously contingent and weird the Taiwanese semiconductor industry's success is.
I'm hesitant to quantify how worthwhile, exactly, but let's give it a BOTEC:
Set our baseline for AGI by the end of 2025 at ~15%; our P(Doom|AGI) can be, maybe, 20%, so we have . Pretty paltry... but it's not inconceivable to say that a Taiwan-invasion would drop AGI-in-2025 odds by a lot: maybe to 1%, so .
Then our Anthropic Factor is , or a 3% price inflation. So instead of 9¢, you should want to pay... 9.27¢.
This is obviously a pretty lame result, but a) at scale, you can expect to make a bit of money and b) if neither invasion nor extinction happens this year, the 2026 market should be somewhat more strongly biased, and the 2027 market even more than that.
By the way, I think it's just as possible for the causation to flow visa-restriction → war as the other way around: I don't remember exactly where I heard this (please point me to the source if you know it!), but I believe the Chinese have a joke that goes something like:
One night, the war cabinet convenes to discuss a nuclear first strike on American soil. They've worked out which missile sites to launch from, how they'll maintain OpSec, how to shelter and defend against a response—the only remaining question is where to strike.
"Washington, DC, of course!" says the Premier.
"No no no, that won't do," says his Defense Minister. "I've got a daughter at Georgetown—my wife would kill me!"
"Ok, how about New York City?"
This time the Chief of the Joint Staff Department pipes up: "Oh, absolutely not, I've got two nephews and a niece at Columbia! Can you imagine what my mother would say?"
Exasperated, the Premier asks, "Boston?"
A half-dozen voices ring out, "No way! Harvard, MIT, Northeastern—are you kidding?"
And so on...
...As long as you don't have any dependents who'll be left alive and made to pay your debts. That's why extinction is such a nice, clean Anthropic outcome: you're made totally even.
The wordplay is unintended, but awesome.
Alright, so, fine, we won the AL Pennant in 2012... with Miguel fucking Cabrera at the height of his powers... and then we also got totally smoked, 4-0, in the Series. Be serious please. This shouldn't be even remotely possible.
I know this sounds silly, mostly because I wrote it with silly-voice, but it's actually just a straightforwardly good and true solution to the problem.
Note that in this case a "Yes" resolution will raise extinction risk, so the Anthropically-Biased price will be somewhat lower than $0.50.
This is probably less of a factor for long-term Manifold markets... Although they've got a much messier model in general, and a thousand other sources of bias, so I'm really not sure how much Anthropic Reasoning will matter over there.