Today's post is a tad gloomier than usual, as I measure such things.  It deals with a thought experiment I invented to smash my own optimism, after I realized that optimism had misled me.  Those readers sympathetic to arguments like, "It's important to keep our biases because they help us stay happy," should consider not reading.  (Unless they have something to protect, including their own life.)

    So!  Looking back on the magnitude of my own folly, I realized that at the root of it had been a disbelief in the Future's vulnerability—a reluctance to accept that things could really turn out wrong.  Not as the result of any explicit propositional verbal belief.  More like something inside that persisted in believing, even in the face of adversity, that everything would be all right in the end.

    Some would account this a virtue (zettai daijobu da yo), and others would say that it's a thing necessary for mental health.

    But we don't live in that world.  We live in the world beyond the reach of God.

    It's been a long, long time since I believed in God.  Growing up in an Orthodox Jewish family, I can recall the last remembered time I asked God for something, though I don't remember how old I was.  I was putting in some request on behalf of the next-door-neighboring boy, I forget what exactly—something along the lines of, "I hope things turn out all right for him," or maybe "I hope he becomes Jewish."

    I remember what it was like to have some higher authority to appeal to, to take care of things I couldn't handle myself.  I didn't think of it as "warm", because I had no alternative to compare it to.  I just took it for granted.

    Still I recall, though only from distant childhood, what it's like to live in the conceptually impossible possible world where God exists.  Really exists, in the way that children and rationalists take all their beliefs at face value.

    In the world where God exists, does God intervene to optimize everything?  Regardless of what rabbis assert about the fundamental nature of reality, the take-it-seriously operational answer to this question is obviously "No".  You can't ask God to bring you a lemonade from the refrigerator instead of getting one yourself.  When I believed in God after the serious fashion of a child, so very long ago, I didn't believe that.

    Postulating that particular divine inaction doesn't provoke a full-blown theological crisis.  If you said to me, "I have constructed a benevolent superintelligent nanotech-user", and I said "Give me a banana," and no banana appeared, this would not yet disprove your statement.  Human parents don't always do everything their children ask.  There are some decent fun-theoretic arguments—I even believe them myself—against the idea that the best kind of help you can offer someone, is to always immediately give them everything they want.  I don't think that eudaimonia is formulating goals and having them instantly fulfilled; I don't want to become a simple wanting-thing that never has to plan or act or think.

    So it's not necessarily an attempt to avoid falsification, to say that God does not grant all prayers.  Even a Friendly AI might not respond to every request.

    But clearly, there exists some threshold of horror awful enough that God will intervene.  I remember that being true, when I believed after the fashion of a child.

    The God who does not intervene at all, no matter how bad things get—that's an obvious attempt to avoid falsification, to protect a belief-in-belief.  Sufficiently young children don't have the deep-down knowledge that God doesn't really exist.  They really expect to see a dragon in their garage.  They have no reason to imagine a loving God who never acts.  Where exactly is the boundary of sufficient awfulness?  Even a child can imagine arguing over the precise threshold.  But of course God will draw the line somewhere.  Few indeed are the loving parents who, desiring their child to grow up strong and self-reliant, would let their toddler be run over by a car.

    The obvious example of a horror so great that God cannot tolerate it, is death—true death, mind-annihilation.  I don't think that even Buddhism allows that.  So long as there is a God in the classic sense—full-blown, ontologically fundamental, the God—we can rest assured that no sufficiently awful event will ever, ever happen.  There is no soul anywhere that need fear true annihilation; God will prevent it.

    What if you build your own simulated universe?  The classic example of a simulated universe is Conway's Game of Life.  I do urge you to investigate Life if you've never played it—it's important for comprehending the notion of "physical law".  Conway's Life has been proven Turing-complete, so it would be possible to build a sentient being in the Life universe, albeit it might be rather fragile and awkward.  Other cellular automata would make it simpler.

    Could you, by creating a simulated universe, escape the reach of God?  Could you simulate a Game of Life containing sentient entities, and torture the beings therein?  But if God is watching everywhere, then trying to build an unfair Life just results in the God stepping in to modify your computer's transistors.  If the physics you set up in your computer program calls for a sentient Life-entity to be endlessly tortured for no particular reason, the God will intervene.  God being omnipresent, there is no refuge anywhere for true horror:  Life is fair.

    But suppose that instead you ask the question:

    Given such-and-such initial conditions, and given such-and-such cellular automaton rules, what would be the mathematical result?

    Not even God can modify the answer to this question, unless you believe that God can implement logical impossibilities.  Even as a very young child, I don't remember believing that.  (And why would you need to believe it, if God can modify anything that actually exists?)

    What does Life look like, in this imaginary world where every step follows only from its immediate predecessor?  Where things only ever happen, or don't happen, because of the cellular automaton rules?  Where the initial conditions and rules don't describe any God that checks over each state?  What does it look like, the world beyond the reach of God?

    That world wouldn't be fair.  If the initial state contained the seeds of something that could self-replicate, natural selection might or might not take place, and complex life might or might not evolve, and that life might or might not become sentient, with no God to guide the evolution.  That world might evolve the equivalent of conscious cows, or conscious dolphins, that lacked hands to improve their condition; maybe they would be eaten by conscious wolves who never thought that they were doing wrong, or cared.

    If in a vast plethora of worlds, something like humans evolved, then they would suffer from diseases—not to teach them any lessons, but only because viruses happened to evolve as well, under the cellular automaton rules.

    If the people of that world are happy, or unhappy, the causes of their happiness or unhappiness may have nothing to do with good or bad choices they made.  Nothing to do with free will or lessons learned.  In the what-if world where every step follows only from the cellular automaton rules, the equivalent of Genghis Khan can murder a million people, and laugh, and be rich, and never be punished, and live his life much happier than the average.  Who prevents it?  God would prevent it from ever actually happening, of course; He would at the very least visit some shade of gloom in the Khan's heart.  But in the mathematical answer to the question What if? there is no God in the axioms.  So if the cellular automaton rules say that the Khan is happy, that, simply, is the whole and only answer to the what-if question.  There is nothing, absolutely nothing, to prevent it.

    And if the Khan tortures people horribly to death over the course of days, for his own amusement perhaps?  They will call out for help, perhaps imagining a God.  And if you really wrote that cellular automaton, God would intervene in your program, of course.  But in the what-if question, what the cellular automaton would do under the mathematical rules, there isn't any God in the system.  Since the physical laws contain no specification of a utility function—in particular, no prohibition against torture—then the victims will be saved only if the right cells happen to be 0 or 1.  And it's not likely that anyone will defy the Khan; if they did, someone would strike them with a sword, and the sword would disrupt their organs and they would die, and that would be the end of that.  So the victims die, screaming, and no one helps them; that is the answer to the what-if question.

    Could the victims be completely innocent?  Why not, in the what-if world?  If you look at the rules for Conway's Game of Life (which is Turing-complete, so we can embed arbitrary computable physics in there), then the rules are really very simple.  Cells with three living neighbors stay alive; cells with two neighbors stay the same, all other cells die.  There isn't anything in there about only innocent people not being horribly tortured for indefinite periods.

    Is this world starting to sound familiar?

    Belief in a fair universe often manifests in more subtle ways than thinking that horrors should be outright prohibited:  Would the twentieth century have gone differently, if Klara Pölzl and Alois Hitler had made love one hour earlier, and a different sperm fertilized the egg, on the night that Adolf Hitler was conceived?

    For so many lives and so much loss to turn on a single event, seems disproportionate.  The Divine Plan ought to make more sense than that.  You can believe in a Divine Plan without believing in God—Karl Marx surely did.  You shouldn't have millions of lives depending on a casual choice, an hour's timing, the speed of a microscopic flagellum.  It ought not to be allowed.  It's too disproportionate.  Therefore, if Adolf Hitler had been able to go to high school and become an architect, there would have been someone else to take his role, and World War II would have happened the same as before.

    But in the world beyond the reach of God, there isn't any clause in the physical axioms which says "things have to make sense" or "big effects need big causes" or "history runs on reasons too important to be so fragile".  There is no God to impose that order, which is so severely violated by having the lives and deaths of millions depend on one small molecular event.

    The point of the thought experiment is to lay out the God-universe and the Nature-universe side by side, so that we can recognize what kind of thinking belongs to the God-universe.  Many who are atheists, still think as if certain things are not allowed.  They would lay out arguments for why World War II was inevitable and would have happened in more or less the same way, even if Hitler had become an architect.  But in sober historical fact, this is an unreasonable belief; I chose the example of World War II because from my reading, it seems that events were mostly driven by Hitler's personality, often in defiance of his generals and advisors.  There is no particular empirical justification that I happen to have heard of, for doubting this.  The main reason to doubt would be refusal to accept that the universe could make so little sense—that horrible things could happen so lightly, for no more reason than a roll of the dice.

    But why not?  What prohibits it?

    In the God-universe, God prohibits it.  To recognize this is to recognize that we don't live in that universe.  We live in the what-if universe beyond the reach of God, driven by the mathematical laws and nothing else.  Whatever physics says will happen, will happen.  Absolutely anything, good or bad, will happen.  And there is nothing in the laws of physics to lift this rule even for the really extreme cases, where you might expect Nature to be a little more reasonable.

    Reading William Shirer's The Rise and Fall of the Third Reich, listening to him describe the disbelief that he and others felt upon discovering the full scope of Nazi atrocities, I thought of what a strange thing it was, to read all that, and know, already, that there wasn't a single protection against it.  To just read through the whole book and accept it; horrified, but not at all disbelieving, because I'd already understood what kind of world I lived in.

    Once upon a time, I believed that the extinction of humanity was not allowed.  And others who call themselves rationalists, may yet have things they trust.  They might be called "positive-sum games", or "democracy", or "technology", but they are sacred.  The mark of this sacredness is that the trustworthy thing can't lead to anything really bad; or they can't be permanently defaced, at least not without a compensatory silver lining.  In that sense they can be trusted, even if a few bad things happen here and there.

    The unfolding history of Earth can't ever turn from its positive-sum trend to a negative-sum trend; that is not allowed.  Democraciesmodern liberal democracies, anyway—won't ever legalize torture.  Technology has done so much good up until now, that there can't possibly be a Black Swan technology that breaks the trend and does more harm than all the good up until this point.

    There are all sorts of clever arguments why such things can't possibly happen.  But the source of these arguments is a much deeper belief that such things are not allowed.  Yet who prohibits?  Who prevents it from happening?  If you can't visualize at least one lawful universe where physics say that such dreadful things happen—and so they do happen, there being nowhere to appeal the verdict—then you aren't yet ready to argue probabilities.

    Could it really be that sentient beings have died absolutely for thousands or millions of years, with no soul and no afterlife—and not as part of any grand plan of Nature—not to teach any great lesson about the meaningfulness or meaninglessness of life—not even to teach any profound lesson about what is impossible—so that a trick as simple and stupid-sounding as vitrifying people in liquid nitrogen can save them from total annihilation—and a 10-second rejection of the silly idea can destroy someone's soul?  Can it be that a computer programmer who signs a few papers and buys a life-insurance policy continues into the far future, while Einstein rots in a grave?  We can be sure of one thing:  God wouldn't allow it.  Anything that ridiculous and disproportionate would be ruled out.  It would make a mockery of the Divine Plan—a mockery of the strong reasons why things must be the way they are.

    You can have secular rationalizations for things being not allowed.  So it helps to imagine that there is a God, benevolent as you understand goodness—a God who enforces throughout Reality a minimum of fairness and justice—whose plans make sense and depend proportionally on people's choices—who will never permit absolute horror—who does not always intervene, but who at least prohibits universes wrenched completely off their track... to imagine all this, but also imagine that you, yourself, live in a what-if world of pure mathematics—a world beyond the reach of God, an utterly unprotected world where anything at all can happen.

    If there's any reader still reading this, who thinks that being happy counts for more than anything in life, then maybe they shouldn't spend much time pondering the unprotectedness of their existence.  Maybe think of it just long enough to sign up themselves and their family for cryonics, and/or write a check to an existential-risk-mitigation agency now and then.  And wear a seatbelt and get health insurance and all those other dreary necessary things that can destroy your life if you miss that one step... but aside from that, if you want to be happy, meditating on the fragility of life isn't going to help.

    But this post was written for those who have something to protect.

    What can a twelfth-century peasant do to save themselves from annihilation?  Nothing.  Nature's little challenges aren't always fair.  When you run into a challenge that's too difficult, you suffer the penalty; when you run into a lethal penalty, you die.  That's how it is for people, and it isn't any different for planets.  Someone who wants to dance the deadly dance with Nature, does need to understand what they're up against:  Absolute, utter, exceptionless neutrality.

    Knowing this won't always save you.  It wouldn't save a twelfth-century peasant, even if they knew.  If you think that a rationalist who fully understands the mess they're in, must surely be able to find a way out—then you trust rationality, enough said.

    Some commenter is bound to castigate me for putting too dark a tone on all this, and in response they will list out all the reasons why it's lovely to live in a neutral universe.  Life is allowed to be a little dark, after all; but not darker than a certain point, unless there's a silver lining.

    Still, because I don't want to create needless despair, I will say a few hopeful words at this point:

    If humanity's future unfolds in the right way, we might be able to make our future light cone fair(er).  We can't modify fundamental physics, but on a higher level of organization we could build some guardrails and put down some padding; organize the particles into a pattern that does some internal checks against catastrophe.  There's a lot of stuff out there that we can't touch—but it may help to consider everything that isn't in our future light cone, as being part of the "generalized past".  As if it had all already happened.  There's at least the prospect of defeating neutrality, in the only future we can touch—the only world that it accomplishes something to care about.

    Someday, maybe, immature minds will reliably be sheltered.  Even if children go through the equivalent of not getting a lollipop, or even burning a finger, they won't ever be run over by cars.

    And the adults wouldn't be in so much danger.  A superintelligence—a mind that could think a trillion thoughts without a misstep—would not be intimidated by a challenge where death is the price of a single failure.  The raw universe wouldn't seem so harsh, would be only another problem to be solved.

    The problem is that building an adult is itself an adult challenge.  That's what I finally realized, years ago.

    If there is a fair(er) universe, we have to get there starting from this world—the neutral world, the world of hard concrete with no padding, the world where challenges are not calibrated to your skills.

    Not every child needs to stare Nature in the eyes.  Buckling a seatbelt, or writing a check, is not that complicated or deadly.  I don't say that every rationalist should meditate on neutrality.  I don't say that every rationalist should think all these unpleasant thoughts.  But anyone who plans on confronting an uncalibrated challenge of instant death, must not avoid them.

    What does a child need to do—what rules should they follow, how should they behave—to solve an adult problem?

    New Comment
    280 comments, sorted by Click to highlight new comments since:
    Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

    I don't think that even Buddhism allows that.
    Depends on the version of Buddhism and who you ask... but yes, even the utter destruction of the mind.

    Of course, 'utter destruction' is not a well-defined term. Depending on who you ask, nothing in Buddhism is ever actually destroyed. Or in the Dust hypothesis, or the Library of Babel... the existence of the mind never ends, because we've never beaten our wives in the first place.

    I live with this awareness.

    "Conway's Life has been proven Turing-complete, so it would be possible to build a sentient being in the Life universe"

    Bit of a leap in logic here, no?

    Read Gödel, Escher, Bach. And google "Turing Machine".

    Worst case, our laws of physics seem to be turing-computable.


    The leap is that the Church–Turing thesis applies to human (“sentient”) cognition. Many theists deny this.

    Many theists deny this... To elaborate, if God exists then consciousness depends on having an immaterial soul. If consciousness depends on an immaterial soul, then simulated entities can never truly be conscious. If the simulated entities aren't really conscious they are incapable of suffering, and there's no reason for God to intervene in the simulation. The thought experiment is not a very effective argument against theism, as it assumes non-existence of souls, but it serves the purpose of illustrating how unthinkably horrible things can actually happen.

    if God exists then consciousness depends on having an immaterial soul.

    I translate that into logical notation:

    (God exists) -> For all X (X is conscious -> X has an immaterial soul)

    I don't concede this conditional. I can imagine a universe with a personal creator, where consciousness is a material property of certain types of complex systems, but souls don't exist.

    Eliezer (I think) feels the same way about the necessity of souls as about the Judeo-Christian god. Interesting hypothesis, but too complex to have anything but a small prior. Then no supporting evidence shows up, despite millennia of looking, reducing the likelihood further.
    Has Eliezer suggested that he believes that the Judeo-Christian god is an "Interesting hypothesis"? My model of him wouldn't say that.
    I think I meant “interesting” in a sarcastic tone. Another way of putting it: “You (theists) claim a high level of belief in this hypothesis. Because so many people (including close family members) take this position, I have though about this hypothesis and find it too complex to have anything but a small prior. Then I asked myself what observations are more likely if the hypothesis is true and which would be less likely. Then I looked around and found no evidence in favor of your hypothesis.”
    4Said Achmiz
    A number of your conditionals are false. This is totally out of nowhere. What has God's existence have to do with what consciousness does or does not depend on? They seem to be entirely logically independent. (This one has already been handled by hwc.) False again, because there's no a priori reason why simulated entities can't have an immaterial soul. (For instance, if God exists and is omnipotent, then by definition he could cause it to be the case that (some or all) simulated entities have immaterial souls.) And false a third time, because it assumes that suffering depends on consciousness. A number of e.g. animal rights proponents deny this.
    Seems to me your comment would have received more votes if you had amplified it a bit considering the majority viewpoint of readers attracted to this blog. What Eli's assumption depends upon: The biblical words are 'God created man in his own image', which hinges on assuming God created the universe. Now, if God can create us in his own image, why can't we create a sentient AI in Our own image? Did god pass on to us whatever "power" he used to endow us with sentience so that we are also empowered to pass on sentience? Can we arrive at a correct answer just be looking at the evidence? From the theistic approach we live in a universe .. one theory (Linde) is that we live in a multiverse with many local universes with their own laws of physics, perhaps they are turing-computable? There is controversy about whether the baby universe is shaped (inherits) laws from the parent universe or whether the physical laws of the baby universe evolve on their own, essentially random in relationship to the parent universe. It is known that experiment cannot provide an answer to which way this unfolds. Looking at the physical laws of this universe (observations) doesn't provide insight as to which, if any, inheritable traits are passed from parent to baby universe. In other words, even if the laws of this universe are turing-computable (Zuse, Fredkin/Wolfram, and Deutsch in an expanded CT Thesis sense) that doesn't provide the foundation for a firm conclusion, because not all possibilities are excluded with this amount of information. Computability is an algorithmic thus cause and effect structure. This doesn't answer the question of whether the origin of the universe is likewise computable. Most current theories introduce faster than light source moments and computability/law of cause and effect, have a speed of light limitation. A similar difficulty arises in the effort to reconcile Relativity and Quantum Theory->to make it universal, called the Problem of Gravity which is really

    "In sober historical fact", clear minds could already see in 1919 that the absurdity of the Treaty of Versailles (with its total ignorance of economic realities, and entirely fueled by hate and revenge) was preparing the next war -- each person (in both nominally winning and nominally defeated countries) being put in such unendurable situations that "he listens to whatever instruction of hope, illusion or revenge is carried to him on the air".

    This was J.M. Keynes writing in 1919, when A. Hitler was working as a police spy for the Rechswehr, infiltrating a tiny party then named DAP (and only later renamed to NDA); Keynes' dire warnings had nothing specifically to do with this "irrelevant" individual, which he had no doubt never even heard about -- there were plenty of other matches ready to set fire to a tinderbox world, after all; for examle, at that time, Benito Mussolini was a much more prominent figure, a well known and controversial journalist, and had just founded the "Fasci Nazionali di Combattimento".

    So your claim, that believing the European errors in 1919 made another great war extremely likely, "is an unreasonable belief",... (read more)

    The claim isn't that Germany would have been perfectly fine, and would never have started a war or done anything else extreme. And the claim is not that Hitler trashed a country that was ticking along happily.

    The claim is that the history of the twentieth century would have gone substantially differently. World War II might not have happened. The tremendous role that Hitler's idiosyncrasies played in directing events, doesn't seem to leave much rational room for determinism here.

    Well, the raise of fascism and anti-Semitism in Europe at that time was wide-spread. It was not just a man. From the Dreyfus affair in France, to Mussolini and Franco, to the heated rivalries between the fascists leagues and the popular in France, ... the whole of Europe after WW1 and unfair Versailles treaty, then the disaster of the 1929 crisis, was a fertile land for all fascist movements. World War II feels much more like a "natural consequence" of previous events (WW1, Versailles treaty, 1929 crisis) and general historical laws (that "populist" politicians thrive when the economical situation is bad), than of a single man. It would have been different with different leaders in the various major countries involved, sure. If Leon Blum helped Republican Spain against Franco instead of letting them stand alone, things could have changed a lot. And many other events could have gone differently - of course, without Hitler, it would have been different. But different enough so WWII wouldn't occur ? Very unlikely to me - not impossible, but very unlikely with only a single turning point.
    Depends on how strictly you define "WWII", for one thing. For example, I've seen it argued that Hitler crippled the Nazi defense strategy to the extent they might well have won without him. Is it still WWII if it's the War for Freedom under the First Glorious Father? Probably. Still ...
    It's a subtle matter, but... you clearly don't really mean determinism here, because you've said a hundred times before how the universe is ultimately deterministic even at the quantum level. Maybe predictability is the word we want. Or maybe it's something else, like fairness or "moral non-neutrality"; it doesn't seem fair that Hitler could have that large an impact by himself, even though there's nothing remotely non-deterministic about that assertion.

    Macroscopic determinism, i.e., the belief that an outcome was not sensitive to small thermal (never mind quantum) fluctuations. If I'm hungry and somebody offers me a tasty hamburger, it's macroscopically determined that I'll say yes in almost all Everett branches; if Zimbabwe starts printing more money, it's macroscopically determined that their inflation rates will rise further.

    The relevant mathematical term is well-posedness, specifically Specifically, the short-term changes are small or at least bounded, though the long term behavior may change drastically.
    Perhapss something along the lines of "stability"? The idea being that small perturbations of input should lead to only small perturbations down the line. ("Stability" isn't really the proper word for that, but I'm not sure what is.)

    Reminds me of this: "Listen, and understand. That terminator is out there. It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead."

    But my question would be: Is the universe of cause and effect really so less safe than the universe of God? At least in this universe, someone who has an evil whim is limited by the laws of cause and effect, e.g. Hitler had to build tanks first, which gave the allies time to prepare. In that other universe, Supreme Being decides he's bored with us and zap we're gone, no rules he has to follow to achieve that outcome.

    So why is relying on the goodness of God safer than relying on the inexorability of cause and effect?

    Short Answer: It's not. Longer Explanation: The way I understand it, the universe of God feels safer because we think of God as like us. In that world, there's a higher being out there. Since we model that being as having similar motivations, desires, etc., we believe that God will also follow some sort of morality and subscribe to basic ideas of fairness. So He'll be compelled to intervene in the case things get too bad. The existence of God also makes you feel less responsible for your fate. For example, if he chooses to smite you, there's nothing you can do. But in a universe of Math, if you don't take action, no higher being is going to step in to hurt/harm you.
    Also, consider that we exist. If there is such a supreme being, then logically, that supreme being does not object to our existence (since we have not yet been smited). Therefore, to said supreme being, our presence is either desirable or irrelevant. If desirable, our presence can be expected to continue; and the human ego will not allow many people to seriously consider ourselves irrelevant, so that option is often simply not considered.

    Given how widespread white nationalism is in America, (i.e. it's a common phenomenon) and how intimately tied to fascism it is, I think that there's a substantial chance that the leader that would have taken Hitler's place would have shared his predilection for ethnic cleansing, even if not world domination.

    "I don't think that even Buddhism allows that."

    Remove whatever cultural or personal contextual trappings you find draped over a particular expression of Buddhism, and you'll find it very clear that Buddhism does "allow" that, or more precisely, un-asks that question.

    As you chip away at unfounded beliefs, including the belief in an essential self (however defined), or the belief that there can be a "problem to solved" independent of a context for its specification, you may arrive at the realization of a view of the world flippe... (read more)

    By the way, I should clarify that my total disagreement with your thesis on WW2 being single-handedly caused by A. Hitler does in no way imply disagreement with your more general thesis. In general I do believe the "until comes steam-engine-time" theory -- that many macro-scale circumstances must be present to create a favorable environment for some revolutionary change; to a lesser degree, I also do think that mostly, when the macro-environment is ripe, one of the many sparks and matches (that are going off all the time, but normally fizz out b... (read more)

    I thought I already knew all this, but this post has made me realize that I've still, deep down, been thinking as you describe - that the universe can't be that unfair, and that the future isn't really at risk. I guess the world seems like a bit scarier of a place now, but I'm sure I'll go back to being distracted by day-to-day life in short order ;).

    As for cryonics, I'm a little interested, but right now I have too many doubts about it and not enough spare money to go out and sign up immediately.

    With all the sci fi brought up here, I think we are familiar with Hitler's Time Travel Exemption Act.

    Ian C., that is half the philosophy of Epicurus in a nutshell: there are no gods, there is no afterlife, so the worst case scenario is not subject to the whims of petulant deities.

    If you want a sufficient response to optimism, consider: is the probability that you will persist forever 1? If not, it is 0. If there is any probability of your annihilation, no matter how small, you will not survive for an infinite amount of time. That is what happens in an i... (read more)

    Not necessarily. If the risk decreases faster than an inverse function (ie. if the risk is less than 1/n for each event, where n is the number of events), there can be a probability between 0 and 1.

    Unless you make one more Horcrux than yesterday each day, that's never going to happen. And there's still the finite, fixed, non-zero chance of the magic widget being destroyed and all of your backups failing simultaneously, or the false vacuum collapsing. Unless you seriously think you can think up completely novel ways to prevent your death at a constantly-accelerating rate, with no duplicates, many of which can avoid hypothetical universe-ending apocalypses. Unless we find a way to escape the known universe, or discover something similarly munchkinneritorial, we're all going to die.

    What's the point of despair? There seems to be a given assumption in the original post that:

    1) there is no protection, universe is allowed to be horrible --> 2)lets despair

    But number 2 doesn't change 1 one bit. This is not a clever argument to disprove number 1. I'm just saying despair is pointless if it changes nothing. It's like when babies cry automatically when something isn't the way they like because they are programmed to by evolution because this reliably attracted the attention of adults. Despairing about the universe will not attract the attention of adults to make it better. We are the only adults, that's it. I would rather reason along the lines of:

    1) there is no protection, universe is allowed to be horrible --> 2)what can I do to make it better

    Agreed with everything else except the part where this is really sad news that's supposed to make us unhappy.

    Agreed. Despair is an unsophisticated response that's not adaptive to the environment in which we're using it - we know how to despair now, it isn't rewarding, and we should learn to do something more interesting that might get us results sooner than "never".
    What's the point of having feelings or emotions at all? Are they not all "pointless"?
    I suggest that you research the difference between instrumental values and terminal values.
    I understand the difference. Perhaps I wasn't clear. You can't just call feelings "pointless" because they don't change anything.
    1Swimmer963 (Miranda Dixon-Luinenburg)
    You could argue that some feelings do change things and have an effect on actions; sometimes in a negative direction (i.e. anger leading to vengeance and war) sometimes in a positive direction (i.e. Gratitude resulting in kindness and help.) Anger in this example can be considered "pointless" not because it has no effect upon the world, but because it's effect is negative and not endorsed intellectually. I think that's the sense in which despair is pointless in the original example. It does have an effect on the world; it results in people NOT taking actions to make things better. You could argue with the use of the word "pointless", I suppose.

    I don't understand the faith in cryonics.

    In a Universe beyond the reach of God, who is to say that the first civilization technologically advanced enough to revive you will not be a "death gives meaning to life" theocracy which has a policy of reviving those who chose to attempt to escape death in order to submit them and their defrosted family members to 1000 years of unimaginable torture followed by execution?

    Sure, there are many reasons to believe such a development is improbable. But you are still rolling those dice in a Universe beyond God's reach, are you not?

    Putting aside the fact that theocracy doesn't really lend itself to technological advancement, the utilitity likelihood of living longer outweighs the (dis)utility likelihood of being tortured for 1000 years.

    Of course you are. It's still a probability game. But Eliezer's contention is that the probabilities for cryonics look good. It's worth rolling the dice.

    Yes very very bad things can happen for little reason. But of course we still want positive arguments to convince us to assign large probabilities to scenarios about which you want us to worry.


    Where is this noirish Eliezer when he's writing about the existence of free will and non-relativist moral truths?

    Don't get bored with the small shit. Cancers, heart disease, stroke, safety engineering, suicidal depression, neurodegenerations, improved cryonic tech. In the next few decades I'm probably going to see most of you die from that shit (and that's if I'm lucky enough to persist as an observer), when you could've done a lot more to prevent it, if you didn't get bored so easily of dealing with the basics.

    Kip, the colors of rationality are crystal, mirror, and glass.

    Robin, fair enough; but conversely no amount of argument will convince someone in zettai daijobu da yo mode.

    For the benefit of those who haven't been following along with Overcoming Bias, I should note that I actually intend to fix the universe (or at least throw some padding atop my local region of it, as disclaimed above) - I'm not just complaining here.

    Hi Eliezer, Sorry, very late to this discussion. I just want to tell you this is exactly how people become conservatives, not in the US politics sense but in the works of Edmund Burke sense, and maybe there is something to learn from there. From about the Age of Enlightenment the Western world is in this optimistic socially experimentating moods, easily casting away old institutions like feudalism, aristocracy, monarchy or limited government, and all this optimism comes from the belief that history has a course, a given, pre-defined direction and Eric Voegelin pointed out it is the secularization of a theistic belief, "immanentizing the eschathon". Burke and others have also pointed out this optimism comes from a belief that "human nature is good". Also in an Enlightenment faith that acting rationally is kind of easy once you learn what your mistakes were. Ugh. Lacking this optimism, many social changes of the last 300 years look kind of brash. In a hindsight it makes more sense that long-standing institutions like aristocratic nobility were better matches to human cognitive biases. OTOH NRx also gets it wrong, because society had to change to cope with changing technology. The changing of military technology alone - gunpowder democratizing war - had to change things around, probably you cannot really have stuff like nobility when knightly armor is useless against muskets etc. This puts us into the uncomfortable position that in the last 300 years both progressives and reactionaries were wrong. The progressives were way too optimistic, the reactionaries did not accept technological change requires social change. So this is the answer I am trying to find out today. Suppose we are in 1700 somewhere in Europe. Unlike them, we do not believe history has a course making it hard for us to screw up social change, do not believe in Providence, do not believe in God wanting to liberate people or even giving them inalienable rights, do not believe human nature is inheren
    That doesn't seerm relevant to EYs comment, and he doesn't hang iout here much anymore, if you want to contact him try Facebook.

    "If you want a sufficient response to optimism, consider: is the probability that you will persist forever 1? If not, it is 0. If there is any probability of your annihilation, no matter how small, you will not survive for an infinite amount of time. That is what happens in an infinite amount of time: everything possible. If all your backup plans can fail at once, even at P=1/(3^^^3), that number will come of eventually with infinite trials." Zubon, this seems to assume that the probabilities in different periods are independent. It could be that... (read more)

    Without Hitler it's likely Ludendorf would have been in charge and things would have been even worse. So perhaps we should be grateful for Hitler!

    I gather there are some Orthodox Jews involved in Holocaust denial and were in Iran for that, but this post gets me to thinking that there should be more of them if they really believe in a benevolent and omnipotent God that won't allow sufficiently horrible things to happen.

    How widespread is white nationalism in America? I would think it's one of the least popular things around, although perhaps I'm taking the Onion too seriously.

    "The standard rebuttal is that evil is Man's own fault,"

    There is no evil. There is neutrality. The universe isn't man's fault; it isn't anyone's fault.

    I'm not at all saddened by these facts. My emotional state is unaltered. It's because I take them neutrally.

    I've experienced severe pain enough to know that A) Torture works. Really. It does. If you don't believe it, try it. It'll be a short lesson. B) Pain is not such a big deal. It's just an avoid-this-at-all-cost -signal. Sure, I'm in agony, sure, I'd hate to remain in a situation where that signal doesn't go away, but it still is just a signal.

    Perhaps as you look at some spot in the sky, they've already - neutrality allowing - tamed neutrality there; made it Friendly.

    We've got a project to finish.

    More parents might let their toddler get hit by a car if they could fix the toddler afterwards.

    There are an awful lot of types of Buddhism. Some allow mind annihilation, and even claim that it should be our goal. Some strains of Epicurianism hold that mind annihilation is a) neutral, and b) better than what all the religions believed in. Some ancient religions seemed to believe in the same awful universal fate as quantum immortality believers do, e.g. eternal degeneration, progressively advanced Alzheimers forever more or less. Adam Smith suggests that... (read more)

    Eight years late reply, but oh well. I think one of the problems with UFAI isn't just human extinction, or even future human suffering. It's that some kinds of UFAI (the paperclip-maximizer comes to mind) could take over our entire future light cone. preventing any future intelligent life (Earth-originating or otherwise) from evolving and finding a better path.

    Good post, but how to deal with this information so that it is not so burdensome: Conway himself, upon creating The Game of Life, didn't believe that the cellular automaton could 'live' indefinitely, but was proven wrong shortly after his games creation by the discovery of the glider gun. We cannot assume that the cards were dealt perfectly and the universe or our existence is infinite, but we can hope that the pattern we have put down will continue to stand the test of time. Belief that we are impervious to extinction or that the universe will not ultimat... (read more)

    I don't understand why the end of the universe bugs people so much. I'll just be happy to make it to next decade, thanks very much. When my IQ rises a few thousand points, I'll consider things on a longer timescale.


    Would Camus agree with you Eliezer?

    What I don't understand is that we live on a planet, where we don't have all people with significant loose change

    A) signing up for cryonics B) super-saturating the coffers of life-extensionists, extinction-risk-reducers, and AGI developers.

    Instead we currently live on a planet, where their combined (probably) trillions of currency units are doing nothing but bloating as 1s and 0s on hard drives.

    Can someone explain why?

    Alas, most people on the planet either: 1. haven't heard of cryonics / useful life extension, 2. don't take it seriously, 3. have serious misunderstandings about it, or 4. reject it for social reasons. I'm timidly optimistic about the next two generations.
    It's pretty straightforward, most people don't believe that cryonics or life-extension techniques have a reasonable chance of success within their lifetimes. As for extinction-risk-reduction, most people doubt that there are serious extinction risks that can feasibly be mitigated. Given those (perhaps misguided beliefs), then what should they spend their money on other than improving their quality of life to the best degree they know how? When the first person is brought back from cryonic sleep and the disease that put them there cured, you can expect an enormous surge of interest. When someone lives to 150 due to them practicing some sort of life-extension technique, there will be a massive interest. As for extinction-risk-reduction, it would take a lot to get people interested, because extinction is something that hasn't happened for what seems like a really long time and we tend to assume dramatic changes are extremely unlikely.
    "trillions of currency units are doing nothing but bloating as 1s and 0s on hard drives" This seems very unlikely. Most people with significant savings have it invested in stocks, bonds, or other investments - that is, they've given it to other people to do something with it that they think will turn a profit. Of the money that is sitting in bank accounts, most of it is lent out, again to people planning to actually do something with it (like build business, build houses, or buy things on credit).

    "What can a twelfth-century peasant do to save themselves from annihilation? Nothing."

    She did something. She passed on a religious meme whose descendents have inspired me, in turn, to pass on the idea that we should engineer a world that can somehow reach backward to save her from annihilation. That may not prove possible, but some possibilities depend on us for their realization.

    A Jewish prophet once wrote something like this: "Behold, I will send you Elijah the prophet before the coming of the great and dreadful day of the Lord: And he sha... (read more)

    Chad: if you seriously think that Turing-completeness does not imply the possibility of sentience, then you're definitely in the wrong place indeed.

    Is there a FAQ or reference somewhere on why or how Turing completeness implies sentience? I know there are some very bright rational people who don't believe turing completeness is enough for sentience (Searle, Penrose), you wouldn't want them active here? (By the way don't make the mistake of thinking " I don't believe turing completeness is sufficient for sentience" is equivalent to " I believe turning completeness is not sufficient for sentience." I don't know either way, but it sure seems that "knowing" is more like religious belief than rational deduction.)
    The basic idea is that a perfect simulation of a physical human mind would be sentient due to the anti-zombie principle. Since all you need for such a simulation is a Turing machine, it follows that any Turing machine could exhibit sentience given the right program.
    Iff the universe is Turing complete. Have we proven that yet?
    I don't think Turing-completeness is sufficient for sentience either, just necessary; this is why I said 'possibility'.
    Why do you think Turing-completeness is necessary for sentience?

    And I do quite fancy well-written, well-researched "alternate history" fiction, such as Turtledove's, so I'd love to read a novel about what happens in 1812 to the fledgling USA if the British are free to entirely concentrate on that war, not distracted by Napoleon's last hurrahs in their backyard, because Napoleon was never around...


    The "War of 1812" was basically an offshoot of the larger Napoleonic Wars; Britain and France were both interfering with the shipping of "neutral" nations, such as the United States, in or... (read more)

    I should note that I actually intend to fix the universe [...]

    I was not aware that the universe was broken. If so, can we get a replacement instead? ;-)

    It is a strange thing. I often feel the impulse to not believe that something would really be possible - usually when talking about existential risks - and I have to make a conscious effort to suppress that feeling, to remind myself that anything the laws of physics allow is possible. (And even then, I often don't succeed - or don't have the courage to entirely allow myself to succeed.)

    A) Torture works. Really. It does. If you don't believe it, try it. It'll be a short lesson.

    That depends on what you're trying to use it for. Torture is very good at getting people to do whatever they believe will stop the torture. For example, it's a good way to get people to confess to whatever you want them to confess to. Torture is a rather poor way to get people to tell you the truth when they have motive to lie and verification is difficult; they might as well just keep saying things at random until they say something that ends the torture.

    Someone who knows the truth you are trying to get from them is very likely relatively early on to try the actual truth to get out of the torture. If you are smart enough to realize that torturing someone who doesn't know the truth, and have the resources to check multiple hypotheses presented by the tortured, and place enough value on getting the truth, torture on the whole is still really effective. I think it is just a hopeful belief to say "torture does not work."

    Consequentialist: Is it a fair universe where the wealthy live forever and the poor die in the relative blink of an eye? It seems hard for our current society to look past that when setting public policy. This doesn't necessarily explain why there isn't more private money put to the purpose, but I think many of the intelligent and wealthy at the present time would see eternal life quests as a millennial long cliche of laughable selfishness and not in tune with leaving a respectable legacy.


    ...Can someone explain why?

    Many people believe in an afterlife... why sign up for cryonics when you're going to go to Heaven when you die?

    That's probably not the explanation, since there are many millions of atheists who heard about cryonics and/or extinction risks. I figure the actual explanation is a combination of conformity, the bystander effect, the tendency to focus on short term problems, and the Silliness Factor.

    I can only speak for myself on this, but wouldn't sign up for cryonics even if it were free, because I don't want to be revived in the future after I'm dead. (Given the choice, I would rather not have existed at all. However, although mine was not a life worth creating, my continued existence will do far less harm than my abrupt death.)

    This is roughly equivalent to stating you don't want to be revived after you fall asleep tonight. If revival from cryosuspension is possible, there is no difference. You want to wake up tomorrow (if you didn't really, there are many easy ways for you to remedy that), therefore you want to wake up from cryonic suspension. You would rather fall asleep tonight than die just before it, therefore you would/should, rationally speaking, take free cryonics.
    Not equivalent. Now, I'm not saying that I, personally wouldn't want to live (for reasons that are no different from any other animal's reasons), but it's really not equivalent. I have people who depend on me now, I have a good chance of making the world a better place because of my existence. I have people who will immediately suffer if I die. In the future, what would be the value of this primitive mind? Taken completely out of my current context like that, would I really be the same person? Sure, I might enjoy myself, but what more would I contribute? I'd just be redundant, or worse, completely would be kind of like making 2 copies of yourself and sending them around, and imagining that this means you've lived twice as much. Nope...each version of you lives one, slightly more redundant life. Again, I'm not saying that I wouldn't want to live in the future. I'm saying that when you go to bed and wake up each night, you lose ~1% of yourself. When you move to a new city or get a new job, you lose 2% of yourself as all your old habits change and are rewritten by new ones. When someone you love and spend all your time with dies, you lose 5% of yourself, and then it gets rewritten with a new relationships or lifestyle. If you lose your job and everyone you knew in life, you lose 10% of yourself as your lifestyle completely and radically alters. If you actually die, you lose 100% of yourself, of course, and then there is no "you" to speak of. And if you wake up one day 1000 years from now with your entire society fundamentally altered, you lose at least 20% of yourself. It'll grow back of course, possibly better than before. Some might even welcome the changes...I think I probably would. But I can totally understand how this might go under someone's threshold for "continuity".

    There's a corallary mystery category which most of you fall into: why are so few smart people fighting, even anonymously, against policy grounded in repugnancy bias that'll likely reduce their persistence odds? Where's the fight against a global ban on reproductive human cloning? Where's the fight to increase legal organ markets? Where's the defense of China's (and other illiberal nations)rights to use prisoners (including political prisoners) for medical experimentation? Until you square aware your own repugnancy bias based inaction, criticisms of that of... (read more)

    To show that hellish scenarios are worth ignoring, you have to show not only that they're improbable, but also that they're improbable enough to overcome the factor (utility of oblivionish scenario - utility of hellish scenario)/(utility of heavenish scenario - utility of oblivionish scenario), which as far as I can tell could be anywhere between tiny and huge.

    As for global totalitarian dictatorships, I doubt they'd last for more than millions of years without something happening to them.


    "why are so few smart people fighting, even anonymously, against policy grounded in repugnancy bias that'll likely reduce their persistence odds?" Here's a MSM citation of Gene Expression today:

    Steve Sailer is also widely read among conservative (and some other) elites, and there's a whole network of anonymous bloggers associated with him.

    "Where's the fight against a global ban on reproductive human cloning?" Such bans have been fought, primarily throu... (read more)

    I can only speak for myself on this, but wouldn't sign up for cryonics even if it were free, because I don't want to be revived in the future after I'm dead.

    I would probably sign up for cryonics if it were free, with a, "do not revive sticker" and detailed data about me so that future brain studiers would have another data point when trying to figure out how it all works.

    I don't wish that I hadn't been born, but I figure I have a part to play a purpose that no one else seems to be doing. Once that has been done, then unless something I see need doing and is important and sufficiently left field for no one else to be doing, I'll just potter along doing random things until I die.

    "I figure I have a part to play a purpose that no one else seems to be doing"

    How do you figure that? Aren't you a materialist? Or do you just mean that you might find a niche to fill that would be satisfying and perhaps meaningful to someone? I'm having trouble finding a non-teleological interpretation of your comment.

    "If you look at the rules for Conway's Game of Life (which is Turing-complete, so we can embed arbitrary computable physics in there), then the rules are really very simple. Cells with three living neighbors stay alive; cells with two neighbors stay the same, all other cells die. There isn't anything in there about only innocent people not being horribly tortured for indefinite periods."

    While I of course I agree with the general sentiment of the post, I don't think this argument works. There is a relevant quote by John McCarthy:

    "In the 195... (read more)

    Doug, Will: There is no fundamental difference between being revived after dying, waking up after going to sleep, or receiving neurotransmitter in a synapse after it was released. There is nothing special about 10^9 seconds as opposed to 10^4 seconds or 10^-4 seconds. Unless, of course, these times figure into your morality, but these are considerations far out of scope of ancestral environments humans evolved in. This is a care where unnatural category meets unnatural circumstances, so figuring out a correct answer is going to be difficult, and relying on intuitively reinforced judgment would be reckless.

    "So invoking them do not give us any more information."

    I do think we get a little: if such constraints exist, they are a property of the patterns themselves, and not a property of the low-level substrate on which they are implemented. If such a thing were true in this world, it would be a property of people and societies, not a metaphysical property. That rules out a lot of religion and magical thinking, and could be a useful heuristic.

    What probability do you guys assign to the god hypothesis being true?

    The main issue with your question is the word "the." There are vastly many possible ways to define the word "god," any one of which could exist or not. But most of those are also individually vastly complicated and exceedingly unlikely to exist unless there is some causal process that brought them into being, which in the eyes of many actual current human believers of a particular version would disqualify them from godhood.

    You can't 'fix the universe'. You can at most change the properties of small parts of reality -- and that can only be accomplished by accepting and acting in accordance to the nature of reality.

    If you don't like the nature of reality, you'd better try to change what you like.

    What probability do you guys assign to the god hypothesis being true?
    Incoherent 'hypotheses' cannot be assigned a probability; they are, so to speak, "not even wrong".

    P(Christian God exists) = vanishingly small. Does that answer your question, random_guy?

    I don't want to sign up for cryonics because I'm afraid I will be revived brain-damaged. But maybe others are worried they will have the social status of a freak in that future society.

    Not that I am willing to sign up for cryonics but I don't see this as a problem. Presumably some monkeys will be placed on ice at some point in the testing of defrosting and you will not be defrosted until they are sure that the defrosting side does not cause brain damage. Also presumably there should be some way of determining if brain damage has occurred before defrosting happens and hopefully no one is defrosted that has brain damage until a way to fix the brain damage has been discovered. I suppose that if the brain damage could be fixed you might lose some important information which does leave the question if you are still you. However if you believe that you are still yourself with the addition of new information, such as is received each day just by living, then you should likewise believe that you will still be yourself if information is lost. Also one of the assumptions of Cryogenics is that the human lifespan will have been greatly expanded so if you have major amnesia from the freezing you can look at it as trading your current life up to the point of freezing for one that is many multiple in length. This is assuming that cryogenics works as intended, of which point I am not convinced of.

    Great post and discussion. Go Team Rational!

    Eliezer, I think there's a slight inconsistency in your message. On the one hand, there are the posts like this, which can basically be summed up as: "Get off your asses, slackers, and go fix the world." This is a message worth repeating many times and in many different ways.

    On the other hand are the "Chosen One" posts. These posts talk about the big gaps in human capabilities - the idea being that some people just have an indefinable "sparkliness" that gives them the power to do inc... (read more)

    "So, what I'd like to see is a discussion of what the rank-and-file members of Team Rational should be doing to help (and I hope that involves more than donating lots of money to SIAI)." How 'rank-and-file' are we talking here? With what skillset, interests, and level of motivation?

    Writing papers like Nick Bostrom's can be valuable:

    I have an analogy: "justice is like cake, it's permitted to exist but someone has to make it".

    Can you be happier sheltering in ignorance? I'm not convinced. I think that's a strategy that only works while you're lucky.

    It is extraordinarily difficult to figure out how to use volunteers. Almost any nonprofit trying to accomplish a skilled-labor task has many more people who want to volunteer their time than they can use. The Foresight Institute has the same problem: People want to donate time instead of money, but it's really, really hard to use volunteers. If you know a solution to this, by all means share.

    I'm surprised by the commenters who cannot conceive of a future life that is more fun than the one they have now - who can't imagine a future they would want to stick around for. Maybe I should bump the priority of the Fun Theory sequence.

    "The Foresight Institute has the same problem: People want to donate time instead of money, but it's really, really hard to use volunteers. If you know a solution to this, by all means share."

    There's always Amazon's Mechanical Turk ( It's an inefficient use of people's time, but it's better than just telling people to go away. If people are reluctant to donate money, you can ask for donations of books- books are actually a fairly liquid asset (

    Eliezer: Does the law allow just setting them to productive but entirely tangential work, and pocketing the profit for SIAI?

    @Hidden: just a "typical" OB reader, for example. I imagine there are lots of readers who read posts like this and say to themselves "Yeah! There's no God! If we want to be saved, we have to save ourselves! But... how...?" Then they wake up the next day and go to their boring corporate programming jobs.

    @pdf23ds: This feels like tunnel vision. Surely the problem SIAI is working on isn't the ONLY problem worth solving.

    @Eliezer: I recognize that it's hard to use volunteers. But members of Team Rational are not herd thinkers. They probabl... (read more)

    The obvious example of a horror so great that God cannot tolerate it, is death - true death, mind-annihilation. I don't think that even Buddhism allows that.
    This is sort of a surprising thing to hear from someone with a Jewish religious background. Jews spend very little attention and energy on the afterlife. (And your picture of Buddhism is simplistic at best, but other people have already dealt with that). I've heard the interesting theory that this stems from a reaction against their Egyptian captors, who were of course obsessed with death and the ... (read more)

    Hopefully posthumans will be a little bit less stubborn in opposition to new scientific ideas.

    "but on the other hand you're essentially saying that if a person is not a Chosen One, there's not much he can really contribute."

    Do you think there aren't at least a few Neos whom Eliezer, and transhumanism in general, hasn't reached and influenced? I'm sure there are many, though I put the upper limit of number of people capable of doing anything worthwhile below 1M (whether they're doing anything is another matter). Perhaps the figure is much lower. But the "luminaries", boy, they are rare.

    Millions of people are capable of hoovering money well in excess of their personal need. Projects aiming for post-humanity only need to target those people to secure unlimited funding.

    "what makes you so damn important that you need to live forever? Get over yourself. After you die, there will be others taking over your work, assuming it was worth doing. Leave some biological and intellectual offspring and shuffle off this mortal coil and give a new generation a chance"

    I vehemently disagree. What makes me so damn important, huh? What makes you so damn unimportant that you're not even giving it a try? The answer to both of these: You, yourself; you make yourself dman important or don't. Importance and significance are self-made. No one can give them to you. You must earn them.

    There are damn important people. Unfortunately most of them were. Think of the joy if you could revive the best minds who've ever walked the earth. If you aren't one of them, try to become one.

    Mtraven: "I truly have trouble understanding why people here think death is so terrible [...] [S]ince we are all hard-core materialists here, let me remind you that the flow of time is an illusion, spacetime is eternal [...]"

    I actually think this one goes the other way. You choose to live right now, rather than killing yourself. Why not consistently affirm that choice across your entire stretch of spacetime?

    "[W]hat makes you so damn important that you need to live forever?"

    Important to whom?

    Even if you're only capable of becoming an average, main sequence star, and not a quasistellar object outshining billions of others, what you must do is to become that star and not remain unlit. Oftentimes those who appear to shine brightly do so only because there's relative darkness around.

    What if Eliezers weren't so damn rare; what if there were 100,000 x "luminaries"; which Eliezer's blog would you read?

    "Important to whom?"
    Important to the development of the universe. It's an open-ended project where we, its sentient part, decide what the rewards are, we decide what's important. I've come to the conclusion that optimizing, understanding, and controlling that which is (existence) asymptotically perfectly, is the most obvious goal. Until we have that figured out, we need to stick around.

    "What if Eliezers weren't so damn rare"

    The weird obsequiousness towards Eliezer makes yet another appearance on OB.

    What the hell is supposed to be worth anything if life isn't?

    Oh, and while I'm stirring up the pot, let me just say that this statement made me laugh: "But members of Team Rational are not herd thinkers." Dude. Self-undermining much?

    Consequentialist: "I've come to the conclusion that optimizing, understanding, and controlling that which is (existence) asymptotically perfectly, is the most obvious goal."

    You haven't been talking to Roko or Richard Hollerith lately, have you?

    "The weird obsequiousness towards Eliezer makes yet another appearance on OB."

    Quite the contrary. I'd prefer it be so that Eliezer is a dime a dozen. It's the relative darkness around that keeps him in the spotlight. Is suspect there's nothing special - in the Von Neumann sense - about this chap, just that I haven't found anyone like him so far. Care to point some others like him?

    Eliezer, if that last comment was in response to mine it is a disappointingly obtuse misinterpretation which doesn't engage with any of the points I made. "Life" is worth something; that doesn't mean that striving for the infinite extension of individual lives should be a priority.

    I'm surprised by the commenters who cannot conceive of a future life that is more fun than the one they have now - who can't imagine a future they would want to stick around for. Maybe I should bump the priority of the Fun Theory sequence.

    I a different type of fun helping people perform a somewhat meaningful* task than I do when I am just hanging out, puzzle solving, adventure sports or going on holiday. I have a little nagging voice asking, "What was the point of that". Which needs to be placated every so often, else the other types of fun los... (read more)

    You gain experience and new neuron connections all the time, do these things not make you to be yourself? If you are not yourself after gaining experience then the "you" that finishes this sentence is not the "you" that started it, may that "you" rest in peace. Further, I wear glasses which thing augments my abilities greatly, do the glasses make me a different "me" then I would be if glasses had not been invented? If not how is it different then adding new neurons to the brain? Further, is learning new things not a meaningful experience to you? If you are required to learn lots of new things shouldn't that make the experience more enticing, especially if one knew one would have the time to both learn whatever one wished and to apply what one had learned.
    Lets do some necromancy here. Im relatevily new to all ths OB and LW stuff, just getting through major sequences (pls excuse my bad English, its not the language i frequently use). Could u point me in the direction of a thread, where "the glorious possibilities of human immortality" are discussed. What i see from the comments here - is the notion to become some sort of ultimately efficient black hole-like eternal information destroyers. Correct me where im wrong, but it is the logical conclusion of minimizing "self" entropy while maximizing information input.
    The Fun Theory Sequence might help, especially the last post on the list. Can't say more without understanding the purpose of your question better. (Also, I didn't write the grandparent comment.)

    Who knows, perhaps there is a deep fundamental fact that it is not possible to implement sentient beings in a universe where the evaluation rules don't enforce fairness. Or, slightly more plausible, it could be impossible to implement sentient tyrants who don't feel a "shade of gloom" when considering what they've done. Neither scenario sounds very plausible, of course.

    The rule of thumb is: if you can imagine it, you can simulate it (because your brain is a simulator). The simulation may not be easy, but at least it's possible.

    You name specific excuses for why life in the future will be bad for you. It sounds like you see the future as a big abandoned factory, where you are a shadow, and the strange mechanisms do their spooky dance. Think instead of what changes could make the future right specifically for you, with tremendous amount of effort applied to this goal. You are just a human, so attention your comfort can get starts far above the order of whole of humanity thinking about every tiny gesture to make you a little bit more comfortable for millions of years, and thinking a... (read more)

    "The claim isn't that Germany would have been perfectly fine, and would never have started a war or done anything else extreme. And the claim is not that Hitler trashed a country that was ticking along happily.

    The claim is that the history of the twentieth century would have gone substantially differently. World War II might not have happened. The tremendous role that Hitler's idiosyncrasies played in directing events, doesn't seem to leave much rational room for determinism here."

    I disagree. Hitler did not departure very far from the general bel... (read more)

    [sorry for ambiguity: thinking for millions of years, not making comfortable for millions of years]

    It sounds like you see the future as a big abandoned factory, where you are a shadow, and the strange mechanisms do their spooky dance.
    I see the future as full of adults, to which I am a useless child. Or if Eliezer gets his way one adult to which I am an embryo. I can't even help with the equivalent of washing up.

    Think instead of what changes could make the future right specifically for you.
    I'd like a future where people were on a level with me, so I could be of some meaningful use.

    However a future without massive disparities of power and knowledge between myself and the inhabitants, would not be able to revive me from cryo sleep.

    So you don't think you could catch up? If you had been frozen somewhere between -10000 and -100 years and revived now, don't you think you could start learning what the heck it is people are doing and understand nowadays? Besides a lot of the pre-freeze life-experience would be fully applicable to present. Everyone starts learning from the point of birth. You'd have headway compared to those who just start out from nothing.
    There are things we can meaningfully contribute to even in a Sysop universe, filled with Minds. We, after all, are minds, too, which h... (read more)

    This is a big do-it-yourself project. Don't complain about there not being enough opportunities to do meaningful things. If you don't find anything meaningful to do, that's your failure, not the failure of the universe. Searching for meaningful problems to solve is part of the project.

    Correction: headway - I meant to say headstart.

    Giant cheesecake fallacy. If future could do everything you wanted to do, it doesn't mean it would do so. Especially if it will be bad for you. If future decides to let you work on a problem, even though it could solve it without you, you can't apply to the uselessness of your action: if future refuses to perform it, only you can make a difference. You can grow to be able to vastly expand the number of things you will be capable of doing, this source never dwindles. If someone or something else solved a problem, it doesn't necessarily spoil the fun for eve... (read more)

    A "head start" in the wrong direction isn't much help.

    Imagine a priest in the temple of Zeus, back in Ancient Greece. Really ancient. The time of Homer, not Archimedes. He makes how best to serve the gods the guiding principle of his life. Now, imagine that he is resurrected in the world of today. What do you think would happen to him? He doesn't speak any modern language. He doesn't know how to use a toilet. He'd freak out at the sight of a television. Nobody worships the gods any more. Our world would seem not only strange, but blasphemous and ... (read more)

    Doug: From almost every perspective I could think of, it would be better to invest resources in raising a newborn than to recreate and rehabilitate a random individual from our barbaric past.

    No, for him it won't be better. Altruistic aspect of the humane morality will help, even if it's more energy-efficient to incinerate you. For that matter, why raise a newborn child instead of making a paperclip?

    In the interest of helping folks here to "overcome bias", I should add just how creepy it is to outside observers to see the unswervingly devoted members of "Team Rational" post four or five comments to each Eliezer post that consist of little more than homilies to his pronouncements, scattered with hyperlinks to his previous scriptural utterances. Some of the more level-headed here like HA have commented on this already. Frankly it reeks of cultism and dogma, the aromas of Ayn Rand, Scientology and Est are beginning to waft from this blog. I think some of you want to live forever so you can grovel and worship Eli for all eternity. . .

    "I think some of you want to live forever so you can grovel and worship Eli for all eternity" The sentence is funnier when one knows that Eli is God in some languages. (not Eliezer but Eli) I would change it to be that reason or rationalism (or Bayesianism) is the object of worship and Eliezer is the Prophet. It certainly makes pointing out errors (in reasoning) in some of the religion posts a less enticing proposition. However, it does seem that not everyone is like that. Also, if actual reason is trusted and not dogmatic assertions that such and such is reasonable then error will eventually give way to truth. I certainly believe Eliezer to be mistaken about some things and missing influential observations about others but, for the most part, what he advocates doing with respect to not shutting down thinking because you disagree with things and other such things is correct. If ones faith in whatever one believes in is so fragile that it can not be questioned then that is precisely when ones faith needs to be questioned. One should discover what ones beliefs actually are and what is essential to those beliefs and then see if the questions are bothersome. If so then one should face the questions head on and figure out why and what they do to ones beliefs.
    I'd like a future where people were on a level with me, so I could be of some meaningful use.

    However a future without massive disparities of power and knowledge between myself and the inhabitants, would not be able to revive me from cryo sleep.

    I already guessed that might be the wish of many people. That's one reason why I would like to acquire the knowledge to deliberately create a single not-person, a Very Powerful Optimization Process. What does it take to not be a person? That is one of those moral questions that runs into empirical confusions. But if I could create a VPOP that did not have subjective experience (or the confusion we name subjective experience), and did not have any pleasure or pain, or valuation of itself, then I think it might be possible to have around a superintelligence that did not, just by its presence, supersede us as an adult; but was nonetheless capable of guarding the maturation of humans into adults, and, a rather lesser problem, capable of reviving cryonics patients.

    If there is anything in there that seems like it should be impossible to understand, then remember that mysteries exist in the map, not in the territory.

    The only thing more diff... (read more)

    A.R.: The standard rebuttal is that evil is Man's own fault, for abusing free will.

    That only excuses moral evil, not natural evil.

    I was not aware that the universe was broken. If so, can we get a replacement instead? ;-)

    Britain is broken, but Cameron's on that case.

    Cameron just made a homeopathy advocate Health Secretary. Maybe the problem was Britain not being broken enough...
    An emergency measure should do as little as possible, and everything necessary;

    The VPOP will abolish, "Good bye," no?

    It will obsolete or profoundly alter the nature of emergency surgery doctors, cancer researchers, fund raisers for cancer research, security services, emergency relief workers, existential risk researchers etc...

    Every person on the planet who is trying to act somewhat like an adult will find they are no longer needed to do what is necessary. It doesn't matter that they are obsoleted by a process rather than a person, they are sti... (read more)

    "Frankly it reeks of cultism and dogma,"

    Oh, I wouldn't worry about that too much; that's a cunning project underway to enbias Eliezer with delusions-of-grandeur bias, smarter-than-thou bias and whatnot.

    Anything to harden our master. :D

    "Chad: if you seriously think that Turing-completeness does not imply the possibility of sentience, then you're definitely in the wrong place indeed."

    gwern: The implication is certainly there and it's one I am sympathetic with, but I'd say its far from proven. The leap in logic there is one that will keep the members of the choir nodding along but is not going to win over any converts. A weak argument is a weak argument, whether you agree with the conclusion reached by that argument -- it's better for the cause if the arguments are held to higher standards.


    "If you want a sufficient response to optimism, consider: is the probability that you will persist forever 1? If not, it is 0."

    You're only correct if the probability is constant with respect to time. Consider, however, that some uncertain events have a non-zero probability even if infinite time passes. For example, random walks in three dimensions (or more) are not guaranteed to meet their origin again, even over infinite time:

    gwern: The implication is certainly there and it's one I am sympathetic with, but I'd say its far from proven.
    1) Consciousness exists. 2) There are no known examples of 'infinite' mathematics in the universe. 3) It is therefore more reasonable to say that consciousness can be constructed with non-infinite mathematics than to postulate that it can't.

    Disagree? Give us an example of a phenomenon that cannot be represented by a Turing Machine, and we'll talk.

    I may hold a different belief but this is certainly a working hypothesis and one that should be explored to the fullest extent possible. That is I am not inclined to believe that we are Turing machines but I could be wrong on this as I do not know it to be the case. The hypothesis that we are Turing machines is one that should be explored as fully as possible. If we are not Turing machines then exploring the hypothesis that we are is worth pursuing as it will get us closer to understanding what it is we are. Turing machines rely on a tape of infinite length at least in conception. I imagine the theory has been looked at with tapes of finite length?
    Eliezer: imagine that you, yourself, live in a what-if world of pure mathematics

    Isn't this true? It seems the simplest solution to "why is there something rather than nothing". Is there any real evidence against our apparently timeless, branching physics being part of a purely mathematical structure? I wouldn't be shocked if the bottom was all Bayes-structure :)

    RI, it shouldn't literally be Bayes-structure because Bayes-structure is about inference is about mind. I have certainly considered the possibility that what-if is all there is; but it's got some problems. Just because what-if is something that humans find deductively compelling does not explain how or why it exists Platonically - to suppose that it is necessary just because you can't find yourself not believing it, hardly unravels the mystery. And much worse, it doesn't explain why we find ourselves in a low-entropy universe rather than a high-entropy ... (read more)