Beyond the Reach of God

by Eliezer Yudkowsky10 min read4th Oct 2008279 comments

126

ReligionCourageWorld Optimization
Frontpage

Today's post is a tad gloomier than usual, as I measure such things.  It deals with a thought experiment I invented to smash my own optimism, after I realized that optimism had misled me.  Those readers sympathetic to arguments like, "It's important to keep our biases because they help us stay happy," should consider not reading.  (Unless they have something to protect, including their own life.)

So!  Looking back on the magnitude of my own folly, I realized that at the root of it had been a disbelief in the Future's vulnerability—a reluctance to accept that things could really turn out wrong.  Not as the result of any explicit propositional verbal belief.  More like something inside that persisted in believing, even in the face of adversity, that everything would be all right in the end.

Some would account this a virtue (zettai daijobu da yo), and others would say that it's a thing necessary for mental health.

But we don't live in that world.  We live in the world beyond the reach of God.

It's been a long, long time since I believed in God.  Growing up in an Orthodox Jewish family, I can recall the last remembered time I asked God for something, though I don't remember how old I was.  I was putting in some request on behalf of the next-door-neighboring boy, I forget what exactly—something along the lines of, "I hope things turn out all right for him," or maybe "I hope he becomes Jewish."

I remember what it was like to have some higher authority to appeal to, to take care of things I couldn't handle myself.  I didn't think of it as "warm", because I had no alternative to compare it to.  I just took it for granted.

Still I recall, though only from distant childhood, what it's like to live in the conceptually impossible possible world where God exists.  Really exists, in the way that children and rationalists take all their beliefs at face value.

In the world where God exists, does God intervene to optimize everything?  Regardless of what rabbis assert about the fundamental nature of reality, the take-it-seriously operational answer to this question is obviously "No".  You can't ask God to bring you a lemonade from the refrigerator instead of getting one yourself.  When I believed in God after the serious fashion of a child, so very long ago, I didn't believe that.

Postulating that particular divine inaction doesn't provoke a full-blown theological crisis.  If you said to me, "I have constructed a benevolent superintelligent nanotech-user", and I said "Give me a banana," and no banana appeared, this would not yet disprove your statement.  Human parents don't always do everything their children ask.  There are some decent fun-theoretic arguments—I even believe them myself—against the idea that the best kind of help you can offer someone, is to always immediately give them everything they want.  I don't think that eudaimonia is formulating goals and having them instantly fulfilled; I don't want to become a simple wanting-thing that never has to plan or act or think.

So it's not necessarily an attempt to avoid falsification, to say that God does not grant all prayers.  Even a Friendly AI might not respond to every request.

But clearly, there exists some threshold of horror awful enough that God will intervene.  I remember that being true, when I believed after the fashion of a child.

The God who does not intervene at all, no matter how bad things get—that's an obvious attempt to avoid falsification, to protect a belief-in-belief.  Sufficiently young children don't have the deep-down knowledge that God doesn't really exist.  They really expect to see a dragon in their garage.  They have no reason to imagine a loving God who never acts.  Where exactly is the boundary of sufficient awfulness?  Even a child can imagine arguing over the precise threshold.  But of course God will draw the line somewhere.  Few indeed are the loving parents who, desiring their child to grow up strong and self-reliant, would let their toddler be run over by a car.

The obvious example of a horror so great that God cannot tolerate it, is death—true death, mind-annihilation.  I don't think that even Buddhism allows that.  So long as there is a God in the classic sense—full-blown, ontologically fundamental, the God—we can rest assured that no sufficiently awful event will ever, ever happen.  There is no soul anywhere that need fear true annihilation; God will prevent it.

What if you build your own simulated universe?  The classic example of a simulated universe is Conway's Game of Life.  I do urge you to investigate Life if you've never played it—it's important for comprehending the notion of "physical law".  Conway's Life has been proven Turing-complete, so it would be possible to build a sentient being in the Life universe, albeit it might be rather fragile and awkward.  Other cellular automata would make it simpler.

Could you, by creating a simulated universe, escape the reach of God?  Could you simulate a Game of Life containing sentient entities, and torture the beings therein?  But if God is watching everywhere, then trying to build an unfair Life just results in the God stepping in to modify your computer's transistors.  If the physics you set up in your computer program calls for a sentient Life-entity to be endlessly tortured for no particular reason, the God will intervene.  God being omnipresent, there is no refuge anywhere for true horror:  Life is fair.

But suppose that instead you ask the question:

Given such-and-such initial conditions, and given such-and-such cellular automaton rules, what would be the mathematical result?

Not even God can modify the answer to this question, unless you believe that God can implement logical impossibilities.  Even as a very young child, I don't remember believing that.  (And why would you need to believe it, if God can modify anything that actually exists?)

What does Life look like, in this imaginary world where every step follows only from its immediate predecessor?  Where things only ever happen, or don't happen, because of the cellular automaton rules?  Where the initial conditions and rules don't describe any God that checks over each state?  What does it look like, the world beyond the reach of God?

That world wouldn't be fair.  If the initial state contained the seeds of something that could self-replicate, natural selection might or might not take place, and complex life might or might not evolve, and that life might or might not become sentient, with no God to guide the evolution.  That world might evolve the equivalent of conscious cows, or conscious dolphins, that lacked hands to improve their condition; maybe they would be eaten by conscious wolves who never thought that they were doing wrong, or cared.

If in a vast plethora of worlds, something like humans evolved, then they would suffer from diseases—not to teach them any lessons, but only because viruses happened to evolve as well, under the cellular automaton rules.

If the people of that world are happy, or unhappy, the causes of their happiness or unhappiness may have nothing to do with good or bad choices they made.  Nothing to do with free will or lessons learned.  In the what-if world where every step follows only from the cellular automaton rules, the equivalent of Genghis Khan can murder a million people, and laugh, and be rich, and never be punished, and live his life much happier than the average.  Who prevents it?  God would prevent it from ever actually happening, of course; He would at the very least visit some shade of gloom in the Khan's heart.  But in the mathematical answer to the question What if? there is no God in the axioms.  So if the cellular automaton rules say that the Khan is happy, that, simply, is the whole and only answer to the what-if question.  There is nothing, absolutely nothing, to prevent it.

And if the Khan tortures people horribly to death over the course of days, for his own amusement perhaps?  They will call out for help, perhaps imagining a God.  And if you really wrote that cellular automaton, God would intervene in your program, of course.  But in the what-if question, what the cellular automaton would do under the mathematical rules, there isn't any God in the system.  Since the physical laws contain no specification of a utility function—in particular, no prohibition against torture—then the victims will be saved only if the right cells happen to be 0 or 1.  And it's not likely that anyone will defy the Khan; if they did, someone would strike them with a sword, and the sword would disrupt their organs and they would die, and that would be the end of that.  So the victims die, screaming, and no one helps them; that is the answer to the what-if question.

Could the victims be completely innocent?  Why not, in the what-if world?  If you look at the rules for Conway's Game of Life (which is Turing-complete, so we can embed arbitrary computable physics in there), then the rules are really very simple.  Cells with three living neighbors stay alive; cells with two neighbors stay the same, all other cells die.  There isn't anything in there about only innocent people not being horribly tortured for indefinite periods.

Is this world starting to sound familiar?

Belief in a fair universe often manifests in more subtle ways than thinking that horrors should be outright prohibited:  Would the twentieth century have gone differently, if Klara Pölzl and Alois Hitler had made love one hour earlier, and a different sperm fertilized the egg, on the night that Adolf Hitler was conceived?

For so many lives and so much loss to turn on a single event, seems disproportionate.  The Divine Plan ought to make more sense than that.  You can believe in a Divine Plan without believing in God—Karl Marx surely did.  You shouldn't have millions of lives depending on a casual choice, an hour's timing, the speed of a microscopic flagellum.  It ought not to be allowed.  It's too disproportionate.  Therefore, if Adolf Hitler had been able to go to high school and become an architect, there would have been someone else to take his role, and World War II would have happened the same as before.

But in the world beyond the reach of God, there isn't any clause in the physical axioms which says "things have to make sense" or "big effects need big causes" or "history runs on reasons too important to be so fragile".  There is no God to impose that order, which is so severely violated by having the lives and deaths of millions depend on one small molecular event.

The point of the thought experiment is to lay out the God-universe and the Nature-universe side by side, so that we can recognize what kind of thinking belongs to the God-universe.  Many who are atheists, still think as if certain things are not allowed.  They would lay out arguments for why World War II was inevitable and would have happened in more or less the same way, even if Hitler had become an architect.  But in sober historical fact, this is an unreasonable belief; I chose the example of World War II because from my reading, it seems that events were mostly driven by Hitler's personality, often in defiance of his generals and advisors.  There is no particular empirical justification that I happen to have heard of, for doubting this.  The main reason to doubt would be refusal to accept that the universe could make so little sense—that horrible things could happen so lightly, for no more reason than a roll of the dice.

But why not?  What prohibits it?

In the God-universe, God prohibits it.  To recognize this is to recognize that we don't live in that universe.  We live in the what-if universe beyond the reach of God, driven by the mathematical laws and nothing else.  Whatever physics says will happen, will happen.  Absolutely anything, good or bad, will happen.  And there is nothing in the laws of physics to lift this rule even for the really extreme cases, where you might expect Nature to be a little more reasonable.

Reading William Shirer's The Rise and Fall of the Third Reich, listening to him describe the disbelief that he and others felt upon discovering the full scope of Nazi atrocities, I thought of what a strange thing it was, to read all that, and know, already, that there wasn't a single protection against it.  To just read through the whole book and accept it; horrified, but not at all disbelieving, because I'd already understood what kind of world I lived in.

Once upon a time, I believed that the extinction of humanity was not allowed.  And others who call themselves rationalists, may yet have things they trust.  They might be called "positive-sum games", or "democracy", or "technology", but they are sacred.  The mark of this sacredness is that the trustworthy thing can't lead to anything really bad; or they can't be permanently defaced, at least not without a compensatory silver lining.  In that sense they can be trusted, even if a few bad things happen here and there.

The unfolding history of Earth can't ever turn from its positive-sum trend to a negative-sum trend; that is not allowed.  Democraciesmodern liberal democracies, anyway—won't ever legalize torture.  Technology has done so much good up until now, that there can't possibly be a Black Swan technology that breaks the trend and does more harm than all the good up until this point.

There are all sorts of clever arguments why such things can't possibly happen.  But the source of these arguments is a much deeper belief that such things are not allowed.  Yet who prohibits?  Who prevents it from happening?  If you can't visualize at least one lawful universe where physics say that such dreadful things happen—and so they do happen, there being nowhere to appeal the verdict—then you aren't yet ready to argue probabilities.

Could it really be that sentient beings have died absolutely for thousands or millions of years, with no soul and no afterlife—and not as part of any grand plan of Nature—not to teach any great lesson about the meaningfulness or meaninglessness of life—not even to teach any profound lesson about what is impossible—so that a trick as simple and stupid-sounding as vitrifying people in liquid nitrogen can save them from total annihilation—and a 10-second rejection of the silly idea can destroy someone's soul?  Can it be that a computer programmer who signs a few papers and buys a life-insurance policy continues into the far future, while Einstein rots in a grave?  We can be sure of one thing:  God wouldn't allow it.  Anything that ridiculous and disproportionate would be ruled out.  It would make a mockery of the Divine Plan—a mockery of the strong reasons why things must be the way they are.

You can have secular rationalizations for things being not allowed.  So it helps to imagine that there is a God, benevolent as you understand goodness—a God who enforces throughout Reality a minimum of fairness and justice—whose plans make sense and depend proportionally on people's choices—who will never permit absolute horror—who does not always intervene, but who at least prohibits universes wrenched completely off their track... to imagine all this, but also imagine that you, yourself, live in a what-if world of pure mathematics—a world beyond the reach of God, an utterly unprotected world where anything at all can happen.

If there's any reader still reading this, who thinks that being happy counts for more than anything in life, then maybe they shouldn't spend much time pondering the unprotectedness of their existence.  Maybe think of it just long enough to sign up themselves and their family for cryonics, and/or write a check to an existential-risk-mitigation agency now and then.  And wear a seatbelt and get health insurance and all those other dreary necessary things that can destroy your life if you miss that one step... but aside from that, if you want to be happy, meditating on the fragility of life isn't going to help.

But this post was written for those who have something to protect.

What can a twelfth-century peasant do to save themselves from annihilation?  Nothing.  Nature's little challenges aren't always fair.  When you run into a challenge that's too difficult, you suffer the penalty; when you run into a lethal penalty, you die.  That's how it is for people, and it isn't any different for planets.  Someone who wants to dance the deadly dance with Nature, does need to understand what they're up against:  Absolute, utter, exceptionless neutrality.

Knowing this won't always save you.  It wouldn't save a twelfth-century peasant, even if they knew.  If you think that a rationalist who fully understands the mess they're in, must surely be able to find a way out—then you trust rationality, enough said.

Some commenter is bound to castigate me for putting too dark a tone on all this, and in response they will list out all the reasons why it's lovely to live in a neutral universe.  Life is allowed to be a little dark, after all; but not darker than a certain point, unless there's a silver lining.

Still, because I don't want to create needless despair, I will say a few hopeful words at this point:

If humanity's future unfolds in the right way, we might be able to make our future light cone fair(er).  We can't modify fundamental physics, but on a higher level of organization we could build some guardrails and put down some padding; organize the particles into a pattern that does some internal checks against catastrophe.  There's a lot of stuff out there that we can't touch—but it may help to consider everything that isn't in our future light cone, as being part of the "generalized past".  As if it had all already happened.  There's at least the prospect of defeating neutrality, in the only future we can touch—the only world that it accomplishes something to care about.

Someday, maybe, immature minds will reliably be sheltered.  Even if children go through the equivalent of not getting a lollipop, or even burning a finger, they won't ever be run over by cars.

And the adults wouldn't be in so much danger.  A superintelligence—a mind that could think a trillion thoughts without a misstep—would not be intimidated by a challenge where death is the price of a single failure.  The raw universe wouldn't seem so harsh, would be only another problem to be solved.

The problem is that building an adult is itself an adult challenge.  That's what I finally realized, years ago.

If there is a fair(er) universe, we have to get there starting from this world—the neutral world, the world of hard concrete with no padding, the world where challenges are not calibrated to your skills.

Not every child needs to stare Nature in the eyes.  Buckling a seatbelt, or writing a check, is not that complicated or deadly.  I don't say that every rationalist should meditate on neutrality.  I don't say that every rationalist should think all these unpleasant thoughts.  But anyone who plans on confronting an uncalibrated challenge of instant death, must not avoid them.

What does a child need to do—what rules should they follow, how should they behave—to solve an adult problem?

126

279 comments, sorted by Highlighting new comments since Today at 10:39 AM
New Comment
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

I don't think that even Buddhism allows that.
Depends on the version of Buddhism and who you ask... but yes, even the utter destruction of the mind.

Of course, 'utter destruction' is not a well-defined term. Depending on who you ask, nothing in Buddhism is ever actually destroyed. Or in the Dust hypothesis, or the Library of Babel... the existence of the mind never ends, because we've never beaten our wives in the first place.

I live with this awareness.

"Conway's Life has been proven Turing-complete, so it would be possible to build a sentient being in the Life universe"

Bit of a leap in logic here, no?

-3[anonymous]11yRead Gödel, Escher, Bach. And google "Turing Machine".

Worst case, our laws of physics seem to be turing-computable.

7hwc9yThe leap is that the Church–Turing thesis applies to human (“sentient”) cognition. Many theists deny this.
1Kyro7yMany theists deny this... To elaborate, if God exists then consciousness depends on having an immaterial soul. If consciousness depends on an immaterial soul, then simulated entities can never truly be conscious. If the simulated entities aren't really conscious they are incapable of suffering, and there's no reason for God to intervene in the simulation. The thought experiment is not a very effective argument against theism, as it assumes non-existence of souls, but it serves the purpose of illustrating how unthinkably horrible things can actually happen.
6hwc7yI translate that into logical notation: I don't concede this conditional. I can imagine a universe with a personal creator, where consciousness is a material property of certain types of complex systems, but souls don't exist.
3hwc7yEliezer (I think) feels the same way about the necessity of souls as about the Judeo-Christian god. Interesting hypothesis, but too complex to have anything but a small prior. Then no supporting evidence shows up, despite millennia of looking, reducing the likelihood further.
1wedrifid7yHas Eliezer suggested that he believes that the Judeo-Christian god is an "Interesting hypothesis"? My model of him wouldn't say that.
1hwc7yI think I meant “interesting” in a sarcastic tone. Another way of putting it: “You (theists) claim a high level of belief in this hypothesis. Because so many people (including close family members) take this position, I have though about this hypothesis and find it too complex to have anything but a small prior. Then I asked myself what observations are more likely if the hypothesis is true and which would be less likely. Then I looked around and found no evidence in favor of your hypothesis.”
3Said Achmiz7yA number of your conditionals are false. This is totally out of nowhere. What has God's existence have to do with what consciousness does or does not depend on? They seem to be entirely logically independent. (This one has already been handled by hwc [http://lesswrong.com/lw/uk/beyond_the_reach_of_god/9bh1].) False again, because there's no a priori reason why simulated entities can't have an immaterial soul. (For instance, if God exists and is omnipotent, then by definition he could cause it to be the case that (some or all) simulated entities have immaterial souls.) And false a third time, because it assumes that suffering depends on consciousness. A number of e.g. animal rights proponents deny this.
-2textonyx7ySeems to me your comment would have received more votes if you had amplified it a bit considering the majority viewpoint of readers attracted to this blog. What Eli's assumption depends upon: The biblical words are 'God created man in his own image', which hinges on assuming God created the universe. Now, if God can create us in his own image, why can't we create a sentient AI in Our own image? Did god pass on to us whatever "power" he used to endow us with sentience so that we are also empowered to pass on sentience? Can we arrive at a correct answer just be looking at the evidence? From the theistic approach we live in a universe .. one theory (Linde) is that we live in a multiverse with many local universes with their own laws of physics, perhaps they are turing-computable? There is controversy about whether the baby universe is shaped (inherits) laws from the parent universe or whether the physical laws of the baby universe evolve on their own, essentially random in relationship to the parent universe. It is known that experiment cannot provide an answer to which way this unfolds. Looking at the physical laws of this universe (observations) doesn't provide insight as to which, if any, inheritable traits are passed from parent to baby universe. In other words, even if the laws of this universe are turing-computable (Zuse, Fredkin/Wolfram, and Deutsch in an expanded CT Thesis sense) that doesn't provide the foundation for a firm conclusion, because not all possibilities are excluded with this amount of information. Computability is an algorithmic thus cause and effect structure. This doesn't answer the question of whether the origin of the universe is likewise computable. Most current theories introduce faster than light source moments and computability/law of cause and effect, have a speed of light limitation. A similar difficulty arises in the effort to reconcile Relativity and Quantum Theory->to make it universal, called the Problem of Gravity which is reall

"In sober historical fact", clear minds could already see in 1919 that the absurdity of the Treaty of Versailles (with its total ignorance of economic realities, and entirely fueled by hate and revenge) was preparing the next war -- each person (in both nominally winning and nominally defeated countries) being put in such unendurable situations that "he listens to whatever instruction of hope, illusion or revenge is carried to him on the air".

This was J.M. Keynes writing in 1919, when A. Hitler was working as a police spy for the Rechswehr, infiltrating a tiny party then named DAP (and only later renamed to NDA); Keynes' dire warnings had nothing specifically to do with this "irrelevant" individual, which he had no doubt never even heard about -- there were plenty of other matches ready to set fire to a tinderbox world, after all; for examle, at that time, Benito Mussolini was a much more prominent figure, a well known and controversial journalist, and had just founded the "Fasci Nazionali di Combattimento".

So your claim, that believing the European errors in 1919 made another great war extremely likely, "is an unreasonable belief",... (read more)

The claim isn't that Germany would have been perfectly fine, and would never have started a war or done anything else extreme. And the claim is not that Hitler trashed a country that was ticking along happily.

The claim is that the history of the twentieth century would have gone substantially differently. World War II might not have happened. The tremendous role that Hitler's idiosyncrasies played in directing events, doesn't seem to leave much rational room for determinism here.

5kilobug9yWell, the raise of fascism and anti-Semitism in Europe at that time was wide-spread. It was not just a man. From the Dreyfus affair in France, to Mussolini and Franco, to the heated rivalries between the fascists leagues and the popular in France, ... the whole of Europe after WW1 and unfair Versailles treaty, then the disaster of the 1929 crisis, was a fertile land for all fascist movements. World War II feels much more like a "natural consequence" of previous events (WW1, Versailles treaty, 1929 crisis) and general historical laws (that "populist" politicians thrive when the economical situation is bad), than of a single man. It would have been different with different leaders in the various major countries involved, sure. If Leon Blum helped Republican Spain against Franco instead of letting them stand alone, things could have changed a lot. And many other events could have gone differently - of course, without Hitler, it would have been different. But different enough so WWII wouldn't occur ? Very unlikely to me - not impossible, but very unlikely with only a single turning point.
6Smokeskin8yBe aware of the hindsight bias: http://en.wikipedia.org/wiki/Hindsight_bias [http://en.wikipedia.org/wiki/Hindsight_bias]
2MugaSofer8yDepends on how strictly you define "WWII", for one thing. For example, I've seen it argued that Hitler crippled the Nazi defense strategy to the extent they might well have won without him. Is it still WWII if it's the War for Freedom under the First Glorious Father? Probably. Still ...
3pnrjulius8yIt's a subtle matter, but... you clearly don't really mean determinism here, because you've said a hundred times before how the universe is ultimately deterministic even at the quantum level. Maybe predictability is the word we want. Or maybe it's something else, like fairness or "moral non-neutrality"; it doesn't seem fair that Hitler could have that large an impact by himself, even though there's nothing remotely non-deterministic about that assertion.

Macroscopic determinism, i.e., the belief that an outcome was not sensitive to small thermal (never mind quantum) fluctuations. If I'm hungry and somebody offers me a tasty hamburger, it's macroscopically determined that I'll say yes in almost all Everett branches; if Zimbabwe starts printing more money, it's macroscopically determined that their inflation rates will rise further.

4shminux8yThe relevant mathematical term is well-posedness [http://en.wikipedia.org/wiki/Well-posed_problem], specifically Specifically, the short-term changes are small or at least bounded, though the long term behavior may change drastically.
2Sniffnoy8yPerhapss something along the lines of "stability"? The idea being that small perturbations of input should lead to only small perturbations down the line. ("Stability" isn't really the proper word for that, but I'm not sure what is.)

Reminds me of this: "Listen, and understand. That terminator is out there. It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead."

But my question would be: Is the universe of cause and effect really so less safe than the universe of God? At least in this universe, someone who has an evil whim is limited by the laws of cause and effect, e.g. Hitler had to build tanks first, which gave the allies time to prepare. In that other universe, Supreme Bei... (read more)

2LawrenceC6yShort Answer: It's not. Longer Explanation: The way I understand it, the universe of God feels safer because we think of God as like us. In that world, there's a higher being out there. Since we model that being as having similar motivations, desires, etc., we believe that God will also follow some sort of morality and subscribe to basic ideas of fairness. So He'll be compelled to intervene in the case things get too bad. The existence of God also makes you feel less responsible for your fate. For example, if he chooses to smite you, there's nothing you can do. But in a universe of Math, if you don't take action, no higher being is going to step in to hurt/harm you.
1CCC6yAlso, consider that we exist. If there is such a supreme being, then logically, that supreme being does not object to our existence (since we have not yet been smited). Therefore, to said supreme being, our presence is either desirable or irrelevant. If desirable, our presence can be expected to continue; and the human ego will not allow many people to seriously consider ourselves irrelevant, so that option is often simply not considered.

Given how widespread white nationalism is in America, (i.e. it's a common phenomenon) and how intimately tied to fascism it is, I think that there's a substantial chance that the leader that would have taken Hitler's place would have shared his predilection for ethnic cleansing, even if not world domination.

"I don't think that even Buddhism allows that."

Remove whatever cultural or personal contextual trappings you find draped over a particular expression of Buddhism, and you'll find it very clear that Buddhism does "allow" that, or more precisely, un-asks that question.

As you chip away at unfounded beliefs, including the belief in an essential self (however defined), or the belief that there can be a "problem to solved" independent of a context for its specification, you may arrive at the realization of a view of the world flippe... (read more)

By the way, I should clarify that my total disagreement with your thesis on WW2 being single-handedly caused by A. Hitler does in no way imply disagreement with your more general thesis. In general I do believe the "until comes steam-engine-time" theory -- that many macro-scale circumstances must be present to create a favorable environment for some revolutionary change; to a lesser degree, I also do think that mostly, when the macro-environment is ripe, one of the many sparks and matches (that are going off all the time, but normally fizz out because the environment is NOT ripe) will tend to start the blaze. But there's nothing "inevitable" here: these are probabilistic, Bayesian beliefs, not "blind faith" on my part. One can look at all available detail and information about each historical situation and come to opine that this or that one follows or deviates from the theory. I just happen to think that WW2 is a particularly blatant example where the theory was followed (as Keynes could already dimly see it coming in '19, and he was NOT the only writer of the time to think that way...!); another equally blatant example is Roman history in the lat... (read more)

I thought I already knew all this, but this post has made me realize that I've still, deep down, been thinking as you describe - that the universe can't be that unfair, and that the future isn't really at risk. I guess the world seems like a bit scarier of a place now, but I'm sure I'll go back to being distracted by day-to-day life in short order ;).

As for cryonics, I'm a little interested, but right now I have too many doubts about it and not enough spare money to go out and sign up immediately.

With all the sci fi brought up here, I think we are familiar with Hitler's Time Travel Exemption Act.

Ian C., that is half the philosophy of Epicurus in a nutshell: there are no gods, there is no afterlife, so the worst case scenario is not subject to the whims of petulant deities.

If you want a sufficient response to optimism, consider: is the probability that you will persist forever 1? If not, it is 0. If there is any probability of your annihilation, no matter how small, you will not survive for an infinite amount of time. That is what happens in an i... (read more)

8faul_sname8yNot necessarily. If the risk decreases faster than an inverse function (ie. if the risk is less than 1/n for each event, where n is the number of events), there can be a probability between 0 and 1.
1wizzwizz41yUnless you make one more Horcrux than yesterday each day, that's never going to happen. And there's still the finite, fixed, non-zero chance of the magic widget being destroyed and all of your backups failing simultaneously, or the false vacuum collapsing. Unless you seriously think you can think up completely novel ways to prevent your death at a constantly-accelerating rate, with no duplicates, many of which can avoid hypothetical universe-ending apocalypses. Unless we find a way to escape the known universe, or discover something similarly munchkinneritorial, we're all going to die.

What's the point of despair? There seems to be a given assumption in the original post that:

1) there is no protection, universe is allowed to be horrible --> 2)lets despair

But number 2 doesn't change 1 one bit. This is not a clever argument to disprove number 1. I'm just saying despair is pointless if it changes nothing. It's like when babies cry automatically when something isn't the way they like because they are programmed to by evolution because this reliably attracted the attention of adults. Despairing about the universe will not attract the attention of adults to make it better. We are the only adults, that's it. I would rather reason along the lines of:

1) there is no protection, universe is allowed to be horrible --> 2)what can I do to make it better

Agreed with everything else except the part where this is really sad news that's supposed to make us unhappy.

1Voltairina9yAgreed. Despair is an unsophisticated response that's not adaptive to the environment in which we're using it - we know how to despair now, it isn't rewarding, and we should learn to do something more interesting that might get us results sooner than "never".
-1Houshalter7yWhat's the point of having feelings or emotions at all? Are they not all "pointless"?
0keen7yI suggest that you research the difference between instrumental values and terminal values.
1Houshalter7yI understand the difference. Perhaps I wasn't clear. You can't just call feelings "pointless" because they don't change anything.
1Swimmer9637yYou could argue that some feelings do change things and have an effect on actions; sometimes in a negative direction (i.e. anger leading to vengeance and war) sometimes in a positive direction (i.e. Gratitude resulting in kindness and help.) Anger in this example can be considered "pointless" not because it has no effect upon the world, but because it's effect is negative and not endorsed intellectually. I think that's the sense in which despair is pointless in the original example. It does have an effect on the world; it results in people NOT taking actions to make things better. You could argue with the use of the word "pointless", I suppose.

I don't understand the faith in cryonics.

In a Universe beyond the reach of God, who is to say that the first civilization technologically advanced enough to revive you will not be a "death gives meaning to life" theocracy which has a policy of reviving those who chose to attempt to escape death in order to submit them and their defrosted family members to 1000 years of unimaginable torture followed by execution?

Sure, there are many reasons to believe such a development is improbable. But you are still rolling those dice in a Universe beyond God's reach, are you not?

4[anonymous]11yPutting aside the fact that theocracy doesn't really lend itself to technological advancement, the utilitity likelihood of living longer outweighs the (dis)utility likelihood of being tortured for 1000 years.

Of course you are. It's still a probability game. But Eliezer's contention is that the probabilities for cryonics look good. It's worth rolling the dice.

Yes very very bad things can happen for little reason. But of course we still want positive arguments to convince us to assign large probabilities to scenarios about which you want us to worry.

Where is this noirish Eliezer when he's writing about the existence of free will and non-relativist moral truths?

Don't get bored with the small shit. Cancers, heart disease, stroke, safety engineering, suicidal depression, neurodegenerations, improved cryonic tech. In the next few decades I'm probably going to see most of you die from that shit (and that's if I'm lucky enough to persist as an observer), when you could've done a lot more to prevent it, if you didn't get bored so easily of dealing with the basics.

Kip, the colors of rationality are crystal, mirror, and glass.

Robin, fair enough; but conversely no amount of argument will convince someone in zettai daijobu da yo mode.

For the benefit of those who haven't been following along with Overcoming Bias, I should note that I actually intend to fix the universe (or at least throw some padding atop my local region of it, as disclaimed above) - I'm not just complaining here.

1[anonymous]6yHi Eliezer, Sorry, very late to this discussion. I just want to tell you this is exactly how people become conservatives, not in the US politics sense but in the works of Edmund Burke sense, and maybe there is something to learn from there. From about the Age of Enlightenment the Western world is in this optimistic socially experimentating moods, easily casting away old institutions like feudalism, aristocracy, monarchy or limited government, and all this optimism comes from the belief that history has a course, a given, pre-defined direction and Eric Voegelin pointed out it is the secularization of a theistic belief, "immanentizing the eschathon". Burke and others have also pointed out this optimism comes from a belief that "human nature is good". Also in an Enlightenment faith that acting rationally is kind of easy once you learn what your mistakes were. Ugh. Lacking this optimism, many social changes of the last 300 years look kind of brash. In a hindsight it makes more sense that long-standing institutions like aristocratic nobility were better matches to human cognitive biases. OTOH NRx also gets it wrong, because society had to change to cope with changing technology. The changing of military technology alone - gunpowder democratizing war - had to change things around, probably you cannot really have stuff like nobility when knightly armor is useless against muskets etc. This puts us into the uncomfortable position that in the last 300 years both progressives and reactionaries were wrong. The progressives were way too optimistic, the reactionaries did not accept technological change requires social change. So this is the answer I am trying to find out today. Suppose we are in 1700 somewhere in Europe. Unlike them, we do not believe history has a course making it hard for us to screw up social change, do not believe in Providence, do not believe in God wanting to liberate people or even giving them inalienable rights, do not believe human nature is inhere
0TheAncientGeek6yThat doesn't seerm relevant to EYs comment, and he doesn't hang iout here much anymore, if you want to contact him try Facebook.

"If you want a sufficient response to optimism, consider: is the probability that you will persist forever 1? If not, it is 0. If there is any probability of your annihilation, no matter how small, you will not survive for an infinite amount of time. That is what happens in an infinite amount of time: everything possible. If all your backup plans can fail at once, even at P=1/(3^^^3), that number will come of eventually with infinite trials." Zubon, this seems to assume that the probabilities in different periods are independent. It could be that... (read more)

Without Hitler it's likely Ludendorf would have been in charge and things would have been even worse. So perhaps we should be grateful for Hitler!

I gather there are some Orthodox Jews involved in Holocaust denial and were in Iran for that, but this post gets me to thinking that there should be more of them if they really believe in a benevolent and omnipotent God that won't allow sufficiently horrible things to happen.

How widespread is white nationalism in America? I would think it's one of the least popular things around, although perhaps I'm taking the Onion too seriously.

"The standard rebuttal is that evil is Man's own fault,"

There is no evil. There is neutrality. The universe isn't man's fault; it isn't anyone's fault.

I'm not at all saddened by these facts. My emotional state is unaltered. It's because I take them neutrally.

I've experienced severe pain enough to know that A) Torture works. Really. It does. If you don't believe it, try it. It'll be a short lesson. B) Pain is not such a big deal. It's just an avoid-this-at-all-cost -signal. Sure, I'm in agony, sure, I'd hate to remain in a situation where that signal doesn't go away, but it still is just a signal.

Perhaps as you look at some spot in the sky, they've already - neutrality allowing - tamed neutrality there; made it Friendly.

We've got a project to finish.

More parents might let their toddler get hit by a car if they could fix the toddler afterwards.

There are an awful lot of types of Buddhism. Some allow mind annihilation, and even claim that it should be our goal. Some strains of Epicurianism hold that mind annihilation is a) neutral, and b) better than what all the religions believed in. Some ancient religions seemed to believe in the same awful universal fate as quantum immortality believers do, e.g. eternal degeneration, progressively advanced Alzheimers forever more or less. Adam Smith suggests that... (read more)

0AnthonyC5yEight years late reply, but oh well. I think one of the problems with UFAI isn't just human extinction, or even future human suffering. It's that some kinds of UFAI (the paperclip-maximizer comes to mind) could take over our entire future light cone. preventing any future intelligent life (Earth-originating or otherwise) from evolving and finding a better path.

Good post, but how to deal with this information so that it is not so burdensome: Conway himself, upon creating The Game of Life, didn't believe that the cellular automaton could 'live' indefinitely, but was proven wrong shortly after his games creation by the discovery of the glider gun. We cannot assume that the cards were dealt perfectly and the universe or our existence is infinite, but we can hope that the pattern we have put down will continue to stand the test of time. Belief that we are impervious to extinction or that the universe will not ultimat... (read more)

I don't understand why the end of the universe bugs people so much. I'll just be happy to make it to next decade, thanks very much. When my IQ rises a few thousand points, I'll consider things on a longer timescale.

Would Camus agree with you Eliezer?

What I don't understand is that we live on a planet, where we don't have all people with significant loose change

A) signing up for cryonics B) super-saturating the coffers of life-extensionists, extinction-risk-reducers, and AGI developers.

Instead we currently live on a planet, where their combined (probably) trillions of currency units are doing nothing but bloating as 1s and 0s on hard drives.

Can someone explain why?

4jasonmcdowell10yAlas, most people on the planet either: 1. haven't heard of cryonics / useful life extension, 2. don't take it seriously, 3. have serious misunderstandings about it, or 4. reject it for social reasons. I'm timidly optimistic about the next two generations.
0[anonymous]10yIt's pretty straightforward, most people don't believe that cryonics or life-extension techniques have a reasonable chance of success within their lifetimes. As for extinction-risk-reduction, most people doubt that there are serious extinction risks that can feasibly be mitigated. Given those (perhaps misguided beliefs), then what should they spend their money on other than improving their quality of life to the best degree they know how? When the first person is brought back from cryonic sleep and the disease that put them there cured, you can expect an enormous surge of interest. When someone lives to 150 due to them practicing some sort of life-extension technique, there will be a massive interest. As for extinction-risk-reduction, it would take a lot to get people interested, because extinction is something that hasn't happened for what seems like a really long time and we tend to assume dramatic changes are extremely unlikely.
1AnthonyC5y"trillions of currency units are doing nothing but bloating as 1s and 0s on hard drives" This seems very unlikely. Most people with significant savings have it invested in stocks, bonds, or other investments - that is, they've given it to other people to do something with it that they think will turn a profit. Of the money that is sitting in bank accounts, most of it is lent out, again to people planning to actually do something with it (like build business, build houses, or buy things on credit).

"What can a twelfth-century peasant do to save themselves from annihilation? Nothing."

She did something. She passed on a religious meme whose descendents have inspired me, in turn, to pass on the idea that we should engineer a world that can somehow reach backward to save her from annihilation. That may not prove possible, but some possibilities depend on us for their realization.

A Jewish prophet once wrote something like this: "Behold, I will send you Elijah the prophet before the coming of the great and dreadful day of the Lord: And he sha... (read more)

Chad: if you seriously think that Turing-completeness does not imply the possibility of sentience, then you're definitely in the wrong place indeed.

0mwengler9yIs there a FAQ or reference somewhere on why or how Turing completeness implies sentience? I know there are some very bright rational people who don't believe turing completeness is enough for sentience (Searle, Penrose), you wouldn't want them active here? (By the way don't make the mistake of thinking " I don't believe turing completeness is sufficient for sentience" is equivalent to " I believe turning completeness is not sufficient for sentience." I don't know either way, but it sure seems that "knowing" is more like religious belief than rational deduction.)
2DSimon9yThe basic idea is that a perfect simulation of a physical human mind would be sentient due to the anti-zombie principle. Since all you need for such a simulation is a Turing machine, it follows that any Turing machine could exhibit sentience given the right program.
1wizzwizz41yIff the universe is Turing complete. Have we proven that yet?
0gwern9yI don't think Turing-completeness is sufficient for sentience either, just necessary; this is why I said 'possibility'.
1MaxNanasy4yWhy do you think Turing-completeness is necessary for sentience?

And I do quite fancy well-written, well-researched "alternate history" fiction, such as Turtledove's, so I'd love to read a novel about what happens in 1812 to the fledgling USA if the British are free to entirely concentrate on that war, not distracted by Napoleon's last hurrahs in their backyard, because Napoleon was never around...

Nitpick:

The "War of 1812" was basically an offshoot of the larger Napoleonic Wars; Britain and France were both interfering with the shipping of "neutral" nations, such as the United States, in or... (read more)

I should note that I actually intend to fix the universe [...]

I was not aware that the universe was broken. If so, can we get a replacement instead? ;-)

It is a strange thing. I often feel the impulse to not believe that something would really be possible - usually when talking about existential risks - and I have to make a conscious effort to suppress that feeling, to remind myself that anything the laws of physics allow is possible. (And even then, I often don't succeed - or don't have the courage to entirely allow myself to succeed.)

A) Torture works. Really. It does. If you don't believe it, try it. It'll be a short lesson.

That depends on what you're trying to use it for. Torture is very good at getting people to do whatever they believe will stop the torture. For example, it's a good way to get people to confess to whatever you want them to confess to. Torture is a rather poor way to get people to tell you the truth when they have motive to lie and verification is difficult; they might as well just keep saying things at random until they say something that ends the torture.

1mwengler9ySomeone who knows the truth you are trying to get from them is very likely relatively early on to try the actual truth to get out of the torture. If you are smart enough to realize that torturing someone who doesn't know the truth, and have the resources to check multiple hypotheses presented by the tortured, and place enough value on getting the truth, torture on the whole is still really effective. I think it is just a hopeful belief to say "torture does not work."

Consequentialist: Is it a fair universe where the wealthy live forever and the poor die in the relative blink of an eye? It seems hard for our current society to look past that when setting public policy. This doesn't necessarily explain why there isn't more private money put to the purpose, but I think many of the intelligent and wealthy at the present time would see eternal life quests as a millennial long cliche of laughable selfishness and not in tune with leaving a respectable legacy.

...Can someone explain why?

Many people believe in an afterlife... why sign up for cryonics when you're going to go to Heaven when you die?

That's probably not the explanation, since there are many millions of atheists who heard about cryonics and/or extinction risks. I figure the actual explanation is a combination of conformity, the bystander effect, the tendency to focus on short term problems, and the Silliness Factor.

I can only speak for myself on this, but wouldn't sign up for cryonics even if it were free, because I don't want to be revived in the future after I'm dead. (Given the choice, I would rather not have existed at all. However, although mine was not a life worth creating, my continued existence will do far less harm than my abrupt death.)

2[anonymous]10yThis is roughly equivalent to stating you don't want to be revived after you fall asleep tonight. If revival from cryosuspension is possible, there is no difference. You want to wake up tomorrow (if you didn't really, there are many easy ways for you to remedy that), therefore you want to wake up from cryonic suspension. You would rather fall asleep tonight than die just before it, therefore you would/should, rationally speaking, take free cryonics.
1someonewrongonthenet8yNot equivalent. Now, I'm not saying that I, personally wouldn't want to live (for reasons that are no different from any other animal's reasons), but it's really not equivalent. I have people who depend on me now, I have a good chance of making the world a better place because of my existence. I have people who will immediately suffer if I die. In the future, what would be the value of this primitive mind? Taken completely out of my current context like that, would I really be the same person? Sure, I might enjoy myself, but what more would I contribute? I'd just be redundant, or worse, completely obsolete...it would be kind of like making 2 copies of yourself and sending them around, and imagining that this means you've lived twice as much. Nope...each version of you lives one, slightly more redundant life. Again, I'm not saying that I wouldn't want to live in the future. I'm saying that when you go to bed and wake up each night, you lose ~1% of yourself. When you move to a new city or get a new job, you lose 2% of yourself as all your old habits change and are rewritten by new ones. When someone you love and spend all your time with dies, you lose 5% of yourself, and then it gets rewritten with a new relationships or lifestyle. If you lose your job and everyone you knew in life, you lose 10% of yourself as your lifestyle completely and radically alters. If you actually die, you lose 100% of yourself, of course, and then there is no "you" to speak of. And if you wake up one day 1000 years from now with your entire society fundamentally altered, you lose at least 20% of yourself. It'll grow back of course, possibly better than before. Some might even welcome the changes...I think I probably would. But I can totally understand how this might go under someone's threshold for "continuity".

There's a corallary mystery category which most of you fall into: why are so few smart people fighting, even anonymously, against policy grounded in repugnancy bias that'll likely reduce their persistence odds? Where's the fight against a global ban on reproductive human cloning? Where's the fight to increase legal organ markets? Where's the defense of China's (and other illiberal nations)rights to use prisoners (including political prisoners) for medical experimentation? Until you square aware your own repugnancy bias based inaction, criticisms of that of... (read more)

To show that hellish scenarios are worth ignoring, you have to show not only that they're improbable, but also that they're improbable enough to overcome the factor (utility of oblivionish scenario - utility of hellish scenario)/(utility of heavenish scenario - utility of oblivionish scenario), which as far as I can tell could be anywhere between tiny and huge.

As for global totalitarian dictatorships, I doubt they'd last for more than millions of years without something happening to them.

HA,

"why are so few smart people fighting, even anonymously, against policy grounded in repugnancy bias that'll likely reduce their persistence odds?" Here's a MSM citation of Gene Expression today: http://www.theglobeandmail.com/servlet/story/LAC.20081004.WORDS04//TPStory/Science

Steve Sailer is also widely read among conservative (and some other) elites, and there's a whole network of anonymous bloggers associated with him.

"Where's the fight against a global ban on reproductive human cloning?" Such bans have been fought, primarily throu... (read more)

I can only speak for myself on this, but wouldn't sign up for cryonics even if it were free, because I don't want to be revived in the future after I'm dead.

I would probably sign up for cryonics if it were free, with a, "do not revive sticker" and detailed data about me so that future brain studiers would have another data point when trying to figure out how it all works.

I don't wish that I hadn't been born, but I figure I have a part to play a purpose that no one else seems to be doing. Once that has been done, then unless something I see need doing and is important and sufficiently left field for no one else to be doing, I'll just potter along doing random things until I die.

"I figure I have a part to play a purpose that no one else seems to be doing"

How do you figure that? Aren't you a materialist? Or do you just mean that you might find a niche to fill that would be satisfying and perhaps meaningful to someone? I'm having trouble finding a non-teleological interpretation of your comment.

"If you look at the rules for Conway's Game of Life (which is Turing-complete, so we can embed arbitrary computable physics in there), then the rules are really very simple. Cells with three living neighbors stay alive; cells with two neighbors stay the same, all other cells die. There isn't anything in there about only innocent people not being horribly tortured for indefinite periods."

While I of course I agree with the general sentiment of the post, I don't think this argument works. There is a relevant quote by John McCarthy:

"In the 195... (read more)

Doug, Will: There is no fundamental difference between being revived after dying, waking up after going to sleep, or receiving neurotransmitter in a synapse after it was released. There is nothing special about 10^9 seconds as opposed to 10^4 seconds or 10^-4 seconds. Unless, of course, these times figure into your morality, but these are considerations far out of scope of ancestral environments humans evolved in. This is a care where unnatural category meets unnatural circumstances, so figuring out a correct answer is going to be difficult, and relying on intuitively reinforced judgment would be reckless.

"So invoking them do not give us any more information."

I do think we get a little: if such constraints exist, they are a property of the patterns themselves, and not a property of the low-level substrate on which they are implemented. If such a thing were true in this world, it would be a property of people and societies, not a metaphysical property. That rules out a lot of religion and magical thinking, and could be a useful heuristic.

What probability do you guys assign to the god hypothesis being true?

0AnthonyC5yThe main issue with your question is the word "the." There are vastly many possible ways to define the word "god," any one of which could exist or not. But most of those are also individually vastly complicated and exceedingly unlikely to exist unless there is some causal process that brought them into being, which in the eyes of many actual current human believers of a particular version would disqualify them from godhood.

You can't 'fix the universe'. You can at most change the properties of small parts of reality -- and that can only be accomplished by accepting and acting in accordance to the nature of reality.

If you don't like the nature of reality, you'd better try to change what you like.

What probability do you guys assign to the god hypothesis being true?
Incoherent 'hypotheses' cannot be assigned a probability; they are, so to speak, "not even wrong".

0AndyCossyleon10yP(Christian God exists) = vanishingly small. Does that answer your question, random_guy?

I don't want to sign up for cryonics because I'm afraid I will be revived brain-damaged. But maybe others are worried they will have the social status of a freak in that future society.

2JohnH10yNot that I am willing to sign up for cryonics but I don't see this as a problem. Presumably some monkeys will be placed on ice at some point in the testing of defrosting and you will not be defrosted until they are sure that the defrosting side does not cause brain damage. Also presumably there should be some way of determining if brain damage has occurred before defrosting happens and hopefully no one is defrosted that has brain damage until a way to fix the brain damage has been discovered. I suppose that if the brain damage could be fixed you might lose some important information which does leave the question if you are still you. However if you believe that you are still yourself with the addition of new information, such as is received each day just by living, then you should likewise believe that you will still be yourself if information is lost. Also one of the assumptions of Cryogenics is that the human lifespan will have been greatly expanded so if you have major amnesia from the freezing you can look at it as trading your current life up to the point of freezing for one that is many multiple in length. This is assuming that cryogenics works as intended, of which point I am not convinced of.

Great post and discussion. Go Team Rational!

Eliezer, I think there's a slight inconsistency in your message. On the one hand, there are the posts like this, which can basically be summed up as: "Get off your asses, slackers, and go fix the world." This is a message worth repeating many times and in many different ways.

On the other hand are the "Chosen One" posts. These posts talk about the big gaps in human capabilities - the idea being that some people just have an indefinable "sparkliness" that gives them the power to do inc... (read more)

"So, what I'd like to see is a discussion of what the rank-and-file members of Team Rational should be doing to help (and I hope that involves more than donating lots of money to SIAI)." How 'rank-and-file' are we talking here? With what skillset, interests, and level of motivation?

Writing papers like Nick Bostrom's can be valuable:

http://www.nickbostrom.com/

I have an analogy: "justice is like cake, it's permitted to exist but someone has to make it".

Can you be happier sheltering in ignorance? I'm not convinced. I think that's a strategy that only works while you're lucky.

It is extraordinarily difficult to figure out how to use volunteers. Almost any nonprofit trying to accomplish a skilled-labor task has many more people who want to volunteer their time than they can use. The Foresight Institute has the same problem: People want to donate time instead of money, but it's really, really hard to use volunteers. If you know a solution to this, by all means share.

I'm surprised by the commenters who cannot conceive of a future life that is more fun than the one they have now - who can't imagine a future they would want to stick around for. Maybe I should bump the priority of the Fun Theory sequence.

"The Foresight Institute has the same problem: People want to donate time instead of money, but it's really, really hard to use volunteers. If you know a solution to this, by all means share."

There's always Amazon's Mechanical Turk (https://www.mturk.com/mturk/welcome). It's an inefficient use of people's time, but it's better than just telling people to go away. If people are reluctant to donate money, you can ask for donations of books- books are actually a fairly liquid asset (http://www.cash4books.net/).

Eliezer: Does the law allow just setting them to productive but entirely tangential work, and pocketing the profit for SIAI?

@Hidden: just a "typical" OB reader, for example. I imagine there are lots of readers who read posts like this and say to themselves "Yeah! There's no God! If we want to be saved, we have to save ourselves! But... how...?" Then they wake up the next day and go to their boring corporate programming jobs.

@pdf23ds: This feels like tunnel vision. Surely the problem SIAI is working on isn't the ONLY problem worth solving.

@Eliezer: I recognize that it's hard to use volunteers. But members of Team Rational are not herd thinkers. They probabl... (read more)

The obvious example of a horror so great that God cannot tolerate it, is death - true death, mind-annihilation. I don't think that even Buddhism allows that.
This is sort of a surprising thing to hear from someone with a Jewish religious background. Jews spend very little attention and energy on the afterlife. (And your picture of Buddhism is simplistic at best, but other people have already dealt with that). I've heard the interesting theory that this stems from a reaction against their Egyptian captors, who were of course obsessed with death and the ... (read more)

Hopefully posthumans will be a little bit less stubborn in opposition to new scientific ideas.

"but on the other hand you're essentially saying that if a person is not a Chosen One, there's not much he can really contribute."

Do you think there aren't at least a few Neos whom Eliezer, and transhumanism in general, hasn't reached and influenced? I'm sure there are many, though I put the upper limit of number of people capable of doing anything worthwhile below 1M (whether they're doing anything is another matter). Perhaps the figure is much lower. But the "luminaries", boy, they are rare.

Millions of people are capable of hoovering money well in excess of their personal need. Projects aiming for post-humanity only need to target those people to secure unlimited funding.

"what makes you so damn important that you need to live forever? Get over yourself. After you die, there will be others taking over your work, assuming it was worth doing. Leave some biological and intellectual offspring and shuffle off this mortal coil and give a new generation a chance"

I vehemently disagree. What makes me so damn important, huh? What makes you so damn unimportant that you're not even giving it a try? The answer to both of these: You, yourself; you make yourself dman important or don't. Importance and significance are self-made. No one can give them to you. You must earn them.

There are damn important people. Unfortunately most of them were. Think of the joy if you could revive the best minds who've ever walked the earth. If you aren't one of them, try to become one.

Mtraven: "I truly have trouble understanding why people here think death is so terrible [...] [S]ince we are all hard-core materialists here, let me remind you that the flow of time is an illusion, spacetime is eternal [...]"

I actually think this one goes the other way. You choose to live right now, rather than killing yourself. Why not consistently affirm that choice across your entire stretch of spacetime?

"[W]hat makes you so damn important that you need to live forever?"

Important to whom?

Even if you're only capable of becoming an average, main sequence star, and not a quasistellar object outshining billions of others, what you must do is to become that star and not remain unlit. Oftentimes those who appear to shine brightly do so only because there's relative darkness around.

What if Eliezers weren't so damn rare; what if there were 100,000 x "luminaries"; which Eliezer's blog would you read?

"Important to whom?"
Important to the development of the universe. It's an open-ended project where we, its sentient part, decide what the rewards are, we decide what's important. I've come to the conclusion that optimizing, understanding, and controlling that which is (existence) asymptotically perfectly, is the most obvious goal. Until we have that figured out, we need to stick around.

"What if Eliezers weren't so damn rare"

The weird obsequiousness towards Eliezer makes yet another appearance on OB.

What the hell is supposed to be worth anything if life isn't?

Oh, and while I'm stirring up the pot, let me just say that this statement made me laugh: "But members of Team Rational are not herd thinkers." Dude. Self-undermining much?

Consequentialist: "I've come to the conclusion that optimizing, understanding, and controlling that which is (existence) asymptotically perfectly, is the most obvious goal."

You haven't been talking to Roko or Richard Hollerith lately, have you?

"The weird obsequiousness towards Eliezer makes yet another appearance on OB."

Quite the contrary. I'd prefer it be so that Eliezer is a dime a dozen. It's the relative darkness around that keeps him in the spotlight. Is suspect there's nothing special - in the Von Neumann sense - about this chap, just that I haven't found anyone like him so far. Care to point some others like him?

Eliezer, if that last comment was in response to mine it is a disappointingly obtuse misinterpretation which doesn't engage with any of the points I made. "Life" is worth something; that doesn't mean that striving for the infinite extension of individual lives should be a priority.

I'm surprised by the commenters who cannot conceive of a future life that is more fun than the one they have now - who can't imagine a future they would want to stick around for. Maybe I should bump the priority of the Fun Theory sequence.

I a different type of fun helping people perform a somewhat meaningful* task than I do when I am just hanging out, puzzle solving, adventure sports or going on holiday. I have a little nagging voice asking, "What was the point of that". Which needs to be placated every so often, else the other types of fun los... (read more)

3JohnH10yYou gain experience and new neuron connections all the time, do these things not make you to be yourself? If you are not yourself after gaining experience then the "you" that finishes this sentence is not the "you" that started it, may that "you" rest in peace. Further, I wear glasses which thing augments my abilities greatly, do the glasses make me a different "me" then I would be if glasses had not been invented? If not how is it different then adding new neurons to the brain? Further, is learning new things not a meaningful experience to you? If you are required to learn lots of new things shouldn't that make the experience more enticing, especially if one knew one would have the time to both learn whatever one wished and to apply what one had learned.
0Exemplis6yLets do some necromancy here. Im relatevily new to all ths OB and LW stuff, just getting through major sequences (pls excuse my bad English, its not the language i frequently use). Could u point me in the direction of a thread, where "the glorious possibilities of human immortality" are discussed. What i see from the comments here - is the notion to become some sort of ultimately efficient black hole-like eternal information destroyers. Correct me where im wrong, but it is the logical conclusion of minimizing "self" entropy while maximizing information input.
0hairyfigment6yThe Fun Theory Sequence [http://lesswrong.com/lw/xy/the_fun_theory_sequence/] might help, especially the last post on the list. Can't say more without understanding the purpose of your question better. (Also, I didn't write the grandparent comment.)

Who knows, perhaps there is a deep fundamental fact that it is not possible to implement sentient beings in a universe where the evaluation rules don't enforce fairness. Or, slightly more plausible, it could be impossible to implement sentient tyrants who don't feel a "shade of gloom" when considering what they've done. Neither scenario sounds very plausible, of course.

The rule of thumb is: if you can imagine it, you can simulate it (because your brain is a simulator). The simulation may not be easy, but at least it's possible.

You name specific excuses for why life in the future will be bad for you. It sounds like you see the future as a big abandoned factory, where you are a shadow, and the strange mechanisms do their spooky dance. Think instead of what changes could make the future right specifically for you, with tremendous amount of effort applied to this goal. You are just a human, so attention your comfort can get starts far above the order of whole of humanity thinking about every tiny gesture to make you a little bit more comfortable for millions of years, and thinking a... (read more)

"The claim isn't that Germany would have been perfectly fine, and would never have started a war or done anything else extreme. And the claim is not that Hitler trashed a country that was ticking along happily.

The claim is that the history of the twentieth century would have gone substantially differently. World War II might not have happened. The tremendous role that Hitler's idiosyncrasies played in directing events, doesn't seem to leave much rational room for determinism here."

I disagree. Hitler did not departure very far from the general bel... (read more)

[sorry for ambiguity: thinking for millions of years, not making comfortable for millions of years]

It sounds like you see the future as a big abandoned factory, where you are a shadow, and the strange mechanisms do their spooky dance.
I see the future as full of adults, to which I am a useless child. Or if Eliezer gets his way one adult to which I am an embryo. I can't even help with the equivalent of washing up.

Think instead of what changes could make the future right specifically for you.
I'd like a future where people were on a level with me, so I could be of some meaningful use.

However a future without massive disparities of power and knowledge between myself and the inhabitants, would not be able to revive me from cryo sleep.

So you don't think you could catch up? If you had been frozen somewhere between -10000 and -100 years and revived now, don't you think you could start learning what the heck it is people are doing and understand nowadays? Besides a lot of the pre-freeze life-experience would be fully applicable to present. Everyone starts learning from the point of birth. You'd have headway compared to those who just start out from nothing.
There are things we can meaningfully contribute to even in a Sysop universe, filled with Minds. We, after all, are minds, too, which h... (read more)

This is a big do-it-yourself project. Don't complain about there not being enough opportunities to do meaningful things. If you don't find anything meaningful to do, that's your failure, not the failure of the universe. Searching for meaningful problems to solve is part of the project.

Correction: headway - I meant to say headstart.


I find Eliezer's (and many of the others here) total and complete obsession with the "God" concept endlessly fascinating. I bet you think about "God" more often than the large majority of the nominally religious. This "God" fellow has seriously pwn3d your wetware. . .

Giant cheesecake fallacy. If future could do everything you wanted to do, it doesn't mean it would do so. Especially if it will be bad for you. If future decides to let you work on a problem, even though it could solve it without you, you can't apply to the uselessness of your action: if future refuses to perform it, only you can make a difference. You can grow to be able to vastly expand the number of things you will be capable of doing, this source never dwindles. If someone or something else solved a problem, it doesn't necessarily spoil the fun for eve... (read more)

A "head start" in the wrong direction isn't much help.

Imagine a priest in the temple of Zeus, back in Ancient Greece. Really ancient. The time of Homer, not Archimedes. He makes how best to serve the gods the guiding principle of his life. Now, imagine that he is resurrected in the world of today. What do you think would happen to him? He doesn't speak any modern language. He doesn't know how to use a toilet. He'd freak out at the sight of a television. Nobody worships the gods any more. Our world would seem not only strange, but blasphemous and ... (read more)

Doug: From almost every perspective I could think of, it would be better to invest resources in raising a newborn than to recreate and rehabilitate a random individual from our barbaric past.

No, for him it won't be better. Altruistic aspect of the humane morality will help, even if it's more energy-efficient to incinerate you. For that matter, why raise a newborn child instead of making a paperclip?

In the interest of helping folks here to "overcome bias", I should add just how creepy it is to outside observers to see the unswervingly devoted members of "Team Rational" post four or five comments to each Eliezer post that consist of little more than homilies to his pronouncements, scattered with hyperlinks to his previous scriptural utterances. Some of the more level-headed here like HA have commented on this already. Frankly it reeks of cultism and dogma, the aromas of Ayn Rand, Scientology and Est are beginning to waft from this blog. I think some of you want to live forever so you can grovel and worship Eli for all eternity. . .

1JohnH10y"I think some of you want to live forever so you can grovel and worship Eli for all eternity" The sentence is funnier when one knows that Eli is God in some languages. (not Eliezer but Eli) I would change it to be that reason or rationalism (or Bayesianism) is the object of worship and Eliezer is the Prophet. It certainly makes pointing out errors (in reasoning) in some of the religion posts a less enticing proposition. However, it does seem that not everyone is like that. Also, if actual reason is trusted and not dogmatic assertions that such and such is reasonable then error will eventually give way to truth. I certainly believe Eliezer to be mistaken about some things and missing influential observations about others but, for the most part, what he advocates doing with respect to not shutting down thinking because you disagree with things and other such things is correct. If ones faith in whatever one believes in is so fragile that it can not be questioned then that is precisely when ones faith needs to be questioned. One should discover what ones beliefs actually are and what is essential to those beliefs and then see if the questions are bothersome. If so then one should face the questions head on and figure out why and what they do to ones beliefs.
I'd like a future where people were on a level with me, so I could be of some meaningful use.

However a future without massive disparities of power and knowledge between myself and the inhabitants, would not be able to revive me from cryo sleep.

I already guessed that might be the wish of many people. That's one reason why I would like to acquire the knowledge to deliberately create a single not-person, a Very Powerful Optimization Process. What does it take to not be a person? That is one of those moral questions that runs into empirical confusions. But if I could create a VPOP that did not have subjective experience (or the confusion we name subjective experience), and did not have any pleasure or pain, or valuation of itself, then I think it might be possible to have around a superintelligence that did not, just by its presence, supersede us as an adult; but was nonetheless capable of guarding the maturation of humans into adults, and, a rather lesser problem, capable of reviving cryonics patients.

If there is anything in there that seems like it should be impossible to understand, then remember that mysteries exist in the map, not in the territory.

The only thing more diff... (read more)

A.R.: The standard rebuttal is that evil is Man's own fault, for abusing free will.

That only excuses moral evil, not natural evil.

I was not aware that the universe was broken. If so, can we get a replacement instead? ;-)

Britain is broken, but Cameron's on that case.

1Delta8yCameron just made a homeopathy advocate Health Secretary. Maybe the problem was Britain not being broken enough...
An emergency measure should do as little as possible, and everything necessary;

The VPOP will abolish, "Good bye," no?

It will obsolete or profoundly alter the nature of emergency surgery doctors, cancer researchers, fund raisers for cancer research, security services, emergency relief workers, existential risk researchers etc...

Every person on the planet who is trying to act somewhat like an adult will find they are no longer needed to do what is necessary. It doesn't matter that they are obsoleted by a process rather than a person, they are sti... (read more)

"Frankly it reeks of cultism and dogma,"

Oh, I wouldn't worry about that too much; that's a cunning project underway to enbias Eliezer with delusions-of-grandeur bias, smarter-than-thou bias and whatnot.

Anything to harden our master. :D

"Chad: if you seriously think that Turing-completeness does not imply the possibility of sentience, then you're definitely in the wrong place indeed."

gwern: The implication is certainly there and it's one I am sympathetic with, but I'd say its far from proven. The leap in logic there is one that will keep the members of the choir nodding along but is not going to win over any converts. A weak argument is a weak argument, whether you agree with the conclusion reached by that argument -- it's better for the cause if the arguments are held to higher standards.

Zubon,

"If you want a sufficient response to optimism, consider: is the probability that you will persist forever 1? If not, it is 0."

You're only correct if the probability is constant with respect to time. Consider, however, that some uncertain events have a non-zero probability even if infinite time passes. For example, random walks in three dimensions (or more) are not guaranteed to meet their origin again, even over infinite time:

gwern: The implication is certainly there and it's one I am sympathetic with, but I'd say its far from proven.
1) Consciousness exists. 2) There are no known examples of 'infinite' mathematics in the universe. 3) It is therefore more reasonable to say that consciousness can be constructed with non-infinite mathematics than to postulate that it can't.

Disagree? Give us an example of a phenomenon that cannot be represented by a Turing Machine, and we'll talk.

0JohnH10yI may hold a different belief but this is certainly a working hypothesis and one that should be explored to the fullest extent possible. That is I am not inclined to believe that we are Turing machines but I could be wrong on this as I do not know it to be the case. The hypothesis that we are Turing machines is one that should be explored as fully as possible. If we are not Turing machines then exploring the hypothesis that we are is worth pursuing as it will get us closer to understanding what it is we are. Turing machines rely on a tape of infinite length at least in conception. I imagine the theory has been looked at with tapes of finite length?
Eliezer: imagine that you, yourself, live in a what-if world of pure mathematics

Isn't this true? It seems the simplest solution to "why is there something rather than nothing". Is there any real evidence against our apparently timeless, branching physics being part of a purely mathematical structure? I wouldn't be shocked if the bottom was all Bayes-structure :)

RI, it shouldn't literally be Bayes-structure because Bayes-structure is about inference is about mind. I have certainly considered the possibility that what-if is all there is; but it's got some problems. Just because what-if is something that humans find deductively compelling does not explain how or why it exists Platonically - to suppose that it is necessary just because you can't find yourself not believing it, hardly unravels the mystery. And much worse, it doesn't explain why we find ourselves in a low-entropy universe rather than a high-entropy ... (read more)

Eliezer,
I'm a little disappointed, frankly. I would have thought you'd be over both God and the Problem of Evil by now. Possibly it goes to show just how difficult it is for people raised as (or by) theists to kill God in themselves.

But possibly you'll get there as you go along. I'd tell you what that was like but I don't know myself yet.

Caledonian:

In an argument that is basically attempting to disprove the existence of God, it seems a little disingenuous to me to include premises that effectively rule out God's existence. If you aren't willing to at least allow the possibility of dualism for the sake of argument, then why bother talking about God at all?

Also, I am not sure what your notion of "infinite" mathematics is about. Can you elaborate or point me to some relevant resources?

No, for him it won't be better.

Well, there's also the perspective of the newborn and the person it grows up into; if we consider that perspective, it probably would prefer that it exists. I don't want The Future to contain "me"; I want it to contain someone better than "me". (Or at least happier, considering that I would prefer to not have existed at all.) And I really doubt that my frozen brain will be of much help to The Future in achieving that goal.

the probabilities for cryonics look good.

They don't have to look good, they just have to beat the probabilities of your mind surviving the alternatives. Current alternatives: cremation, interment, scattering over your favourite football pitch. Currently I'm wavering between cryonics and Old Trafford.

Eliezer, I'm ridiculously excited about the next fifty years, and only slightly less excited about the fun theory sequence. Hope it chimes with my own.

Excellent post, agree with every single line of it. It's not depressing for me -- I went through that depression earlier, after finally understanding evolution.

One nitpick -- I find the question at the end of the text redundant.

We already know that all this world around us is just an enormous pattern arising out of physically determined interactions between particles, with no 'essence of goodness' or other fundamental forces of this kind.

So the answer to your question seems obvious to me -- if we don't like patterns we see around us (including us ourselve... (read more)

On the existential question of our pointless existence in a pointless universe, my perspective tends to oscillate between two extremes:

1.) In the more pessimistic (and currently the only rationally defensible) case, I view my mind and existence as just a pattern of information processing on top of messy organic wetware and that is all 'I' will ever be. Uploading is not immortality, it's just duplicating that specific mind pattern at that specific time instance. An epsilon unit of time after the 'upload' event that mind pattern is no longer 'me' and will ... (read more)

RE: the reek of cultishness and dogma, I agree.

Regardless of whether you want to argue that being in a cult might be ok or not anything to worry about, the fact is this sort of thing doesn't look good to other people. You're going to win many converts -- at least the kind you want -- by continuing to put on quasi-religious, messianic airs, and welcoming the sort of fawning praise that seems to come up a lot in the comments here. There's obviously some sharp thinking going on in these parts, but you guys need to pay a bit more attention to your PR.

Not going to win, that should read.

by continuing to put on quasi-religious, messianic airs

Huh. Let a guy have a bit of poetic license now and then, eh? I really don't see what you mean.

The request that we should 'fix the world' suggests that a.)we know that it is broken and b.)we know how to fix it; I am not so sure that this is the case. When one says 'X is wrong/unfair/undesirable etc., one is more often than not actually making a statement about one's state of mind rather than the state of reality i.e., one is saying 'I think or feel that X is wrong/unfair/undesirable'. Personally, I don't like to see images of suffering and death but I'm not sure that my distaste for suffering and death is enough to confidently assert that they are w... (read more)

Alice, can't tell crap from great? Don't worry, 90% of people share your inability. Why? Because 90% of everything is crap. (Sturgeon's law)

Lets fix the things that are obviously crap first. After that, well address the iffy things.

Towards less crappy greatness.

You've said the bit about Paul Graham twice now in this thread; do you actually consider that good reasoning, or are you merely being flip? Paul Graham's followers may or may not be cultish to some degree, but that doesn't bear on the question of whether your own promotional strategies are sound ones. Let me put it this way: you will need solid, technically-minded, traditionally-trained scientists and engineers in your camp if you ever hope to do the things you want to do. The mainstream science community, as a matter of custom, doesn't look favorably upon... (read more)

I take your point...if your point is 'we gotta start somewhere'. Nontheless, the use of 'obviously' is problematic and misleading. To whom is it obvious? To you? Or perhaps you and your friends, or you and other people on the internet who tend to think in the same way as you and with whom you generally agree? Don't get me wrong, I have a very clear idea of what I think is crap (and I strongly suspect it'd be similar to yours) and I'm just as keen to impose my vision of the 'uncrap' on the world as the next person. However, I can't help but be troubled by t... (read more)

Whoops! I thought that comment had been swallowed by the ether, so I said it again. Turns out it's on the previous page. Dup has been deleted.

On the unfairness of existence:

Those who (want to) understand and are able, joyously create things that have always existed as potentials.

Those who don't (want to) understand and can't do anything real, make stuff up that never was possible and never will be.

The former last forever in eternal glory, spanning geological timescales and civilizations, for the patterns they create are compatible with the structure of the universe and sustained by it, while oblivion is reserved for the latter.

Science. The real stuff. Be all you can be.

Science. Keepin' it real.

Science. Live a life with a purpose.
Science. Live a life worth living.

In an argument that is basically attempting to disprove the existence of God, it seems a little disingenuous to me to include premises that effectively rule out God's existence.

How exactly can you construct a disproof of X without using premises that rule out X? That's what disproving is.

Non-infinite mathematics: also known as finite mathematics, also known as discrete mathematics. Non-continuum. Not requiring the existence of the real numbers.

To the best of our knowledge, reality only seems to require the integers, although constructing models that... (read more)

0JohnH10yYou can accept X as a premise and come to a contradiction of X with other accepted premises. Coming to something that seems absurd may also be grounds for doubting X, but doesn't disprove X. It might also be possible to prove that X and ~X are consistent with the other premises, which if the desire is to disprove X should be enough to safely ignore the possibility X is correct without further information. I think for the Turing Machine part of this P/NP would need to be resolved first so he would also win 1 million dollars (or if P=NP then depending on his preferences he might not want to publish and use his code to solve every open question out there and get himself pretty much as much money as he wished)
1khafra10yCaledonian was a combination contrarian and curmudgeon back in the OvercomingBias days, and hasn't been around in years; so you probably won't get a direct reply. However, if I understand this comment correctly as a follow-up to this one [http://lesswrong.com/lw/uk/beyond_the_reach_of_god/3zu3], you may want to look into the Church-Turing Thesis. The theory "physics is computable" is still somewhat controversial, but it has a great deal of support. If physics is computable, and humans are made out of physics, then by the Church-Turing Thesis, humans are Turing Machines.
-1JohnH10yI am actually familiar with the Church-Turing Thesis, as well as both Godel's incompleteness proof and the Halting problem. The theory that humans are Turing machines is one that needs to be investigated.
0AnthonyC5y'The theory that humans are Turing machines is one that needs to be investigated." Yes, but that question isn't where we need to start necessarily. It is a subset of a possibly much simpler problem: are the laws of physics Turing computable? If so, then humans cannot do anything Turing machines cannot do. if not, then the human-specific question remains open. We don't know, and there are many relevant-but-not-too-convincing arguments either way. The laws of physics as generally taught are both continuous (symmetries, calculus in QM), and quantum (discrete allowed particle states, Planck length, linear algebra in QM). No one has ever observed nature calculating with an uncomputable general real (or complex) variable (how could we with human minds and finitely precise instrumentation?), while any computable algebraic or transcendental number seems to be fair game. But, building a model of physics that rules out general real variables is apparently much more difficult. Even if there are general real variables in physics, they may only arise as a result of previous general real variables, in which case whatever-the-universe-runs-on may be able to handle them symbolically instead of explicitly. Anyone here who decides to read my many-years-late wall of text have any idea what the implications would be of this one? Possibly may or may not allow construction of architectures that are not Turing computing but also not fully general, limited by whatever non-Turing-computable stuff happens to have always existed? If space and time are quantized (digital), that makes for more even trouble with special and general relativity - are the Planck length/time/etc.somehow reference frame dependent? Also, see http://lesswrong.com/lw/h9c/can_somebody_explain_this_to_me_the_computability/ [http://lesswrong.com/lw/h9c/can_somebody_explain_this_to_me_the_computability/]
is creating a child after the right and proper fashion of creating a child. If I do not wish to be a father, I think that it is acceptable for me to avoid it; and certainly the skill to deliberately not create a child would be less than the skill to deliberately create a healthy child.

I would agree, I am not trying to create a child either. I'm trying to create brain stuff, and figure out how to hook it up to a human so that it becomes aligned to that humans brain. Admittedly it is giving more power to children, but I think the only feasible way to get... (read more)

>How exactly can you construct a disproof of X without using
>premises that rule out X? That's what disproving is.

Sure, a mathematical proof proceeds from its premises and therefore any results achieved are entailed in those premises. I am not sure we are really in the real of pure mathematics here but I probably should have been more precise in my statement. In a non-mathematical discussion, a slightly longer chain of reasoning is generally preferred -- starting with the premise that dualism is false is a little uncomfortably close to starting with ... (read more)

"Bayesian cult encourages religious people to kill God in themselves" - how's that for a newspaper headline?

P.S. I'd delete this comment after a certain amount of time, you might not want it to get cached by google or something.

The various silly people who think I want to keep the flesh around forever, or constrain all adults to the formal outline of an FAI, are only, of course, making things up; their imagination is not wide enough to understand the concept of some possible AIs being people, and some possible AIs being something else.
Presuming that I am one of these "silly people": Quite the opposite, and it is hard for me to imagine how you could fail to understand that from reading my comments. It is because I can imagine these things, and see that they have impor... (read more)

it doesn't explain why we find ourselves in a low-entropy universe rather than a high-entropy one

I didn't think it would solve all our questions, I just wondered if it was both the simplest solution and lacking good evidence to the contrary. Would there be a higher chance of being a Boltzmann brain in a universe identical to ours that happened to be part of a what-if-world? If not, how is all this low-entropy around me evidence against it?

Just because what-if is something that humans find deductively compelling does not explain how or why it exists Pla
... (read more)
How would our "Block Universe" look different from the inside if it was a what-if-Block-Universe? It all adds up to...

I'm not saying this is wrong, but in its present form, isn't it really a mysterious answer to a mysterious question? If you believed it, would the mystery seem any less mysterious?

suggests that you want to personally live on beyond the Singularity; whereas more coherent interpretations of your ideas that I've heard from Mike Vassar imply annihilation or equivalent transformation of all of us by the day after it
Oops. I really should clarify that Mike didn't mention annihilation. That's my interepretation/extrapolation.

Eliezer, doesn't "math mysteriously exists and we live in it" have one less mystery than "math mysteriously exists and the universe mysteriously exists and we live in it"? (If you don't think math exists it seems like you run into indispensability arguments.)

IIRC the argument for a low-entropy universe is anthropic, something like "most non-simple universes with observers in them look like undetectably different variants of a simple universe rather than universes with dragons in them".

Alastair Malcolm:

in any comparison of all possible combinations of bit/axiom strings up to any equal finite (long) length (many representing not only a world but also (using 'spare' string segments inside the total length) extraneous features such as other worlds, nothing in particular, or perhaps 'invisible' intra-world entities), it is reasonable to suppose that the simplest worlds (ie those with the shortest representing string segments) will occur most often across all strings, since they will have more 'spare' irrelevant bit/axiom combinations up to t... (read more)

Re: The way you present this, as well as the discussion in the comments, suggests you think "death" is a thing that can be avoided by living indefinitely [...]

Er... ;-) Many futurists seem to have it in for death. Bostrom, Kurzweil, Drexler, spring to mind. To me, the main problem seems to be uncopyable minds. If we could change our bodies like a suit of clothes, the associated problems would mostly go away. We will have copyable minds once they are digital.

I'm not saying this is wrong, but in its present form, isn't it really a mysterious answer to a mysterious question? If you believed it, would the mystery seem any less mysterious?

Hmm. You're right.

Darn.

Re: The way you present this, as well as the discussion in the comments, suggests you think "death" is a thing that can be avoided by living indefinitely [...]

Er... ;-) Many futurists seem to have it in for death. Bostrom, Kurzweil, Drexler, spring to mind. To me, the main problem seems to be uncopyable minds. If we could change our bodies like a suit of clothes, the associated problems would mostly go away. We will have copyable minds once they are digital.


"Death" as we know it is a concept that makes sense only because we have cl... (read more)

That sounds like the "One Big Organism" concept. Nick Bostrom has also written about that - e.g. see his What is a Singleton?

The fictional Borg work similarly, I believe. Death would become rather like cutting your toenails.

Phil: [. . .] In such a world, how would anybody know if "you" had died?
Perhaps anyone else knowing whether you're alive or dead wouldn't matter. You die when you lose sufficient component magnitudes and claim strengths on your components. If you formulate the sufficient conditions, you know what counts as death for your decisions, thus for you. If you formulate the sufficiency also as instance in a greater network, you and others know what counts as death for you. In either case, unless you're dying to be suicidally abstract, you're somebody and you know what it means for you to die.

Tim -

What I described involves some similar ideas, but I find the notion of a singleton unlikely, or at least suboptimal. It is a machine analogy for life and intelligence. A machine is a collection of parts, all working together under one common control to one common end. Living systems, by contrast, and particularly large evolving systems such as ecosystems or economies, work best, in our experience, if they do not have centralized control, but have a variety of competing agents, and some randomness.

There are a variety of proposals floating about for ... (read more)

>To exist is to be imperfect
A thing that that philosophical types like to do that I dislike is making claims about about what it is to exist in general, claims that presumably would apply to all minds or 'subjects', when in fact those claims concern at most only the particular Homo Sapiens condition, and are based only on the experiences of one particular Homo Sapiens.

>However, I can't help but be troubled by the thought that the
>mass murder of jews, gypsies, the mentally retarted and
>homosexuals was precipitated by the fact that Hitler et... (read more)

A thing that that philosophical types like to do that I dislike is making claims about about what it is to exist in general, claims that presumably would apply to all minds or 'subjects', when in fact those claims concern at most only the particular Homo Sapiens condition, and are based only on the experiences of one particular Homo Sapiens.

My claim is mainly based on physics of one sort of another. For one the second law of thermodynamics. All systems will eventually degrade to whatever is most stable. Neutrons, IIRC. And unless a set of neutrons in the... (read more)

What I described involves some similar ideas, but I find the notion of a singleton unlikely, or at least suboptimal. It is a machine analogy for life and intelligence. A machine is a collection of parts, all working together under one common control to one common end. Living systems, by contrast, and particularly large evolving systems such as ecosystems or economies, work best, in our experience, if they do not have centralized control, but have a variety of competing agents, and some randomness.

The idea of one big organism is not really that it will b... (read more)

In theory, competition looks very bad. Fighting with each other can't possibly be efficient. Almost always, battles should be done under simulation - so the winner can be determined early - without the damage and waste of a real fight. There's a huge drive towards cooperation - as explained by Robert Wright.
We're talking about competition between optimization processes. What would it mean to be a simulation of a computation? I don't think there is any such distinction. Subjectivity belongs to these processes; and they are the things which must compete.... (read more)

If the winner could be determined by a simpler computation, you would be running that computation instead [...]

Well, that's the point. Usually it can be, and often we're not. There's a big drive towards virtualising combat behaviour in nature. Deer snort at each other, sea lions bellow - and so on: signalling who is going to win without actually fighting. Humans do the same thing with national sports - and with companies - where a virtual creature dies, and the people mostly walk away. But we are still near the beginning of the curve. There are still many fights, and a lot of damage done. Huge improvements in this area could be made.

Tim - I'm asking the question whether competition, and its concomitant unpleasantness (losing, conflict, and the undermining of CEV's viability), can be eliminated from the world. Under a wide variety of assumptions, we can characterize all activities, or at least all mental activities, as computational. We also hope that these computations will be done in a way such that consciousness is still present.

My argument is that optimization is done best by an architecture that uses competition. The computations engaged in this competition are the major possib... (read more)

Alex, I admit I hope the fawning praisers, who are mostly anonymous, are Eliezer's sockpuppets. Rather than a dozen or more people on the internet who read Eliezer's posts and feel some desire to fawn. But it's mostly an aesthetic preference -I can't say it makes a real difference in accomplishing shared goals, beyond being a mild waste of time and energy.

So. Hard. To. Say. Anything. Bad. About. Eliezer.

:)

Aren't you a bit biased here? If one expresses positive views about Eliezer, that's fawning, obsequiousness, or other rather exaggerated word, but negative views and critique is just business as usual. As usual.

It would be better if talking about people ceased and ideas and actions got 100% attention.

Remove the talk about people from politics and what's left? Policies? I don't know what the people/policies ratio in political discussion in the media is, but often it feels like most of the time is spent on talking about the politicians, not about policies. I guess it's supposed to be that way.

My argument is that optimization is done best by an architecture that uses competition.

Optimization is done best by an architecture that performs trials, inspects the results, makes modifications and iterates. No sentient agents typically need to be harmed during such a process - nor do you need multiple intelligent agents to perform it.

Remember the old joke: "Why is there only one Monopolies Commission?"

The evidence for the advantages of cooperation is best interpreted as a lack of our ability to manage large complex structures effectively. ... (read more)

Optimization is done best by an architecture that performs trials, inspects the results, makes modifications and iterates. No sentient agents typically need to be harmed during such a process - nor do you need multiple intelligent agents to perform it.

Some of your problems will be so complicated, that each trial will be undertaken by an organization as complex as a corporation or an entire nation.

If these nations are non-intelligent, and non-conscious, or even unemotional, and incorporate no such intelligences in themselves, then you have a dead world de... (read more)

Ha, I get it now, FAI is about creating god.

Anyways, no matter what you do, mind annihilation is certain in our universe, i.e. 2nd law of thermodynamics.

@Doug S. Read "The Gentle Seduction" by Marc Stiegler. And, if you haven't already, consider anti-depressants: I know a number of people whom they have saved from suicide.

The main question here is -

Can finite state machine have qualia?

Because suffering is qualia. If it is not, it does''t matter. It is easy to write a programm that print "i am suffering" if you press a button.

So, God cannot influence on result of work of final automata. But He can give it qualia, or switch qualia off for it - and nobody would ever mention it.

So, existing of final automat Universe, don''t prove absense of God, because God could change qualia without changing result of work of the automat.

So, God cannot influence on result of work of final automata. But He can give it qualia, or switch qualia off for it - and nobody would ever mention it.

What makes you think qualia aren't necessarily bound to algorithms?

Why the hangup about turing-completeness?

In a finite universe world there are no true turing machines, as there are no infinite tapes; thus if you are going to be assigning some philosophical heft to turing-completeness you are being a bit sloppy, and should be saying "show me something that provably cannot be computed by a finite state machine of any size".

Yes, some Buddhist sects allow for complete annihilation of self. Most of the Zen sects, actually. No gods, no afterlife, just here-and-now, whichever now you happen to be considering. Reincarnation is simply the reconfiguration of you, moment by moment, springing up from the Void (or the quantum foam, if you prefer), each moment separate and distinct from the previous or the subsequent. Dogen (and Bankei and Huineng, for that matter) understood the idea of Timeless Physics very well.

What makes you think qualia aren't necessarily bound to algorithms?
What makes you think that 'qualia' are a meaningful concept?

I have never come across anyone who could present a coherent and intelligible definition for the word that didn't automatically render the referent non-existent.

Before we try to answer the question, we need to establish that the question is a valid one. 'How many angels can dance on the head of a pin?' is not one of the great mysteries, because the question is only meaningful in a context of specific, unjustifiable beliefs. Eliminate those beliefs and there's no more question.

Note: I'm an atheist who, like you, agrees that there's no divine plan and that, good or bad, shit happens.

That said, I think there's a hole in your argument. You're convincing when you claim that unfair things happen on Earth; you're not convincing when you claim there's no afterlife where Earthly-unfairness is addressed.

Isn't that the whole idea (and solace) of the afterlife? (Occam's Razor stops me from believing in an afterlife, but you don't delve into that in your essay.) A theist could easily agree with most of your essay but say, "Don't worry... (read more)

So you don't think you could catch up? If you had been frozen somewhere between -10000 and -100 years and revived now, don't you think you could start learning what the heck it is people are doing and understand nowadays? Besides a lot of the pre-freeze life-experience would be fully applicable to present. Everyone starts learning from the point of birth. You'd have headway compared to those who just start out from nothing.
There are things we can meaningfully contribute to even in a Sysop universe, filled with Minds. We, after all, are minds, too, which h
... (read more)

Minds don't have the "inherent quality of creativity". Autistics are the obvious counterexample.

Sorry, but no, no, no.

The evidence for the advantages of cooperation is best interpreted as a lack of our ability to manage large complex structures effectively. We are so bad at it that even a stupid evolutionary algorithm can do better - despite all the duplication and wasted effort that so obviously involves. Companies that develop competing products to fill a niche in ignorance of each other's efforts often is the stupid waste of time that it seems. In the future, our management skills will improve.

This is the argument for communism. Why should we resurrect it? What con

... (read more)

Some of your problems will be so complicated, that each trial will be undertaken by an organization as complex as a corporation or an entire nation.

Maybe - but test failures are typically not a sign that you need to bin the offending instance. Think of how programmers work. See some unit test failures? Hit undo a few times, until they go away again. Rarely do you need to recycle on the level of worms.

I find it strange how atheists always feel able to speak for God. Can you speak for your human enemies? Can you even speak for your wife, if you have one? Why would you presume to think you can say what God would or wouldn't allow?

Abe: I find it strange how atheists always feel able to speak for God.
Sometimes, they're not trying to speak for God, as they're not first assuming that an ideally intelligent God exists. Rather, they're imagining and speaking about the theist assumption that an ideally intelligent God exists, and then they carefully draw inferences which tend to end up incoherent on that grounding. However, philosophy of religion reasonably attempts coherence, and not all atheists are completely indifferent toward it.

Qualia are neural representations of certain data.
That is NOT what that word is generally used to refer to.
So what? I don't see why so called reductionists quibble over this so much.
Probably because you're using the word incorrectly, so you don't understand what they're objecting to.

That is NOT what that word is generally used to refer to.

Why, because it's a meaningful definition - and people are generally referring to something utterly meaningless? If you want me to define what people, in general, are talking about then of course I can't give a meaningful definition.

But I contend that this is meaningful, and it is what people are referring to - even if they don't know how to properly talk about it.

Imagine person A says that negative numbers are not even conceptually possible, or that arithmetic or whatever can't be performed with the... (read more)

What makes you think that 'qualia' are a meaningful concept?

The problem, of course, is that qualia (or more generally, experiencing-ness) is not a concept at all (well there is a concept of experiencing-ness, but that is just the concept, not the actuality). A metaphor for experiencing-ness is the "theater of awareness" in which all concepts, sensations, and emotions appear and are witnessed. But experiencing-ness is prior to any and all concepts.

Nate Barna: Sometimes, they're not trying to speak for God, as they're not first assuming that an ideally intelligent God exists. Rather, they're imagining and speaking about the theist assumption that an ideally intelligent God exists, and then they carefully draw inferences which tend to end up incoherent on that grounding. However, philosophy of religion reasonably attempts coherence, and not all atheists are completely indifferent toward it.

It may be true that some times atheists carefully draw inferences from the idea of an ideally intelligent God. I... (read more)

Abe: Why would a being that can create minds at will flinch at their annihilation? The absolute sanctity of minds, even before God, is the sentiment of modern western man, not a careful deduction based on an inconceivably superior intelligence.
An atheist can imagine God having the thought: As your God, I don't care that you deny Me. Your denial of Me is inconsequential and unimpressive in the greater picture necessarily inaccessible to you. If this is an ad hoc imagining, then your assumption, in your question, that a being who can create minds at will doesn't flinch at their annihilation must also be ad hoc.

Following on to the sub-thread here, initiated by Recovering Irrationalist, about whether mathematical existence may be all there is, and that we live in it.

What does that say about the title of the post, Beyond the Reach of God?

Wouldn't it imply that there are those who are indeed beyond God's reach, since even God Himself cannot change the nature of mathematics? That is, God does not really have any control over the multiverse; it exists in an invariant form independent of God's existence.

However we can also argue that there are worlds within the multive... (read more)

@Doug S. Read "The Gentle Seduction" by Marc Stiegler. And, if you haven't already, consider anti-depressants: I know a number of people whom they have saved from suicide.

::Googles "The Gentle Seduction"::

Yeah, that's a very beautiful story. And yes, I take antidepressants. They just change my feelings, not my beliefs. Their subjective effect on me can best be described as "Yes, my life still sucks, but I'm cheerful anyway!" If I honestly prefer retroactive non-existence even when happy, doesn't that suggest that my assessment... (read more)

Nothing really matters,
Anyone can see,
Nothing really matters-,nothing really matters to me,

Any way the wind blows....

**
Queen - Bohemian Rhapsody
We miss you Freddie....

I don't understand why you believe this thought exercise leads to despair or unhappiness. I went through this thought experiment many years ago, and the only significant impact it had on me was that I evaluate risk very differently than most people around me. I'm no less happy (or more depressed at least) or motivated, and I experience about as much despair as a non-secular optimist: occasional brief glimpses of it which quickly evaporate as I consider my options and choices.

And, to be honest, the process of looking at existentialism and going through some... (read more)

Lots of ideas here. They only seem to work if God is primarily concerned about fairness on earth. What if God is not so concerned about our circumstance as He in our response to circumstance. After all, He has an eternal perspective, while our perspective is limited by what we see of life. If this were true, then earth, and our existence, are like a big machine designed specifically to sort out the few good from the bad. Being raised in an Orthodox Jewish family, I’m sure you encountered countless examples in the bible where bad stuff happened to go... (read more)

0JohnH10yIn particular as an Orthodox Jew he should be very familiar with Deuteronomy, Isaiah, and Jeremiah where the scattering of the Jews and there centuries of oppression and persecution are predicted as well as there eventual gathering after they have reached the point that they thought they were going to be forgotten forever and utterly destroyed, such that the survivors of the horrors are predicted to say at the number of them, these where did they come from? for we were left alone to paraphrase Isaiah 49:21.

I tend to resolve these issues with measure-problem hand-waving. Basically, since any possible universe exists (between quantum branching, inflationary multiverse, and simulated/purely mathematical existence), any collection of particles (such as me sitting here) exists with a practically uncountable set of futures and pasts, many of which make no sense (bolzman brains). The measure problem is, why is that "many" not actually "most"? The simplest answer is the anthropic one: because that kind of existence simply "doesn't count"... (read more)

[-][anonymous]11y -1

or maybe "I hope he becomes Jewish."

That is a rotten thing to wish upon any adult male. Think of the pain!

Last night I was reading through your "coming of age" articles and stopped right before this one, which neatly summarizes why I was physically terrified. I've never before experienced sheer existential terror, just from considering reality.

Absolute, utter, exceptionless neutrality.

I hate these filthy Neutrals, Kif. With enemies you know where they stand but with Neutrals, who knows? It sickens me.

[-][anonymous]9y 4

Are there any useful summaries of strategies to rearrange priorities and manage time to deal with the implications of this post? I get the existential terror part. We're minds in constant peril, basically floating on a tattered raft in the middle of the worst hurricane ever imagined. I'm sure only few of the contributors here think that saying, "this sucks but oh well" is a good idea. So what do we do?

Since I've started reading LW, I have started to devote way more of my life to reading. I read for hours each day now, mostly science literature, p... (read more)

1Delta8yI realise it is over a year later but can I ask how it went, or whether anyone has advice for someone in a similar position? I felt similar existential terror when reading The Selfish Gene and realising on one level I'm just a machine that will someday break down, leaving nothing behind. How do you respond to something like that? I get that you need to strike a balance between being sufficiently aware of your fragility and mortality to drive yourself to do things that matter (ideally supporting measures that will reduce said human fragility) but not so much you obsess over it and become depressed, but it can seem a pretty tricky balance to strike, especially if you are temperamentally inclined towards obsessiveness, negativity and akrasia.
0[anonymous]8yThere's lots to say, but I'll reserve it for a full discussion post soon, and I'll come back here and post a link.
0Delta8ySounds good, I'll look forward to it.

Not every child needs to stare Nature in the eyes. Buckling a seatbelt, or writing a check, is not that complicated or deadly. I don't say that every rationalist should meditate on neutrality. I don't say that every rationalist should think all these unpleasant thoughts. But anyone who plans on confronting an uncalibrated challenge of instant death, must not avoid them.

Granted. Now, where are the useful, calibrated challenges? I am like a school-age child in my rationality; I can read and understand this passage about neutrality and think about it f... (read more)

I wrote this when I was like sixteen, before I'd ever heard of LessWrong:

Dreaming of things yet to come, bleeding ink into the sand,
we passed the days on hither shore, for time measureless to man;
and soon our bloody mark was made on the sand on which we ran.

Footprints upon the sunless shore were rarer than the stains,
the moon cast light upon the trees, the moon gave us its rain,
and to each other in the night we shouted our refrain:

“We immortal run here, waiting; we don’t die despite the bleeding;
we will continue their lives’ taking if they won’t take our

... (read more)

Thank you, Eliezer. I will cherish this article. People build their entire world views to run from this and here you are depicting the profound brutality - not obscured with fluff, but stripped naked by your words.

a world beyond the reach of God, an utterly unprotected world where anything at all can happen. ... Someone who wants to dance the deadly dance with Nature, does need to understand what they're up against: Absolute, utter, exceptionless neutrality. ... challenges are not calibrated to your skills

I feel a great relief reading these simple ... (read more)

I have never really had a problem with the complete neutrality of life. It doesn't really change what happens, since it's inevitable. I think there is a certain art to learning and not let what you learn consume you with despair. If there is no inherent meaning of life and it is all just what happens, so what? It won't really change anything about your life or how you live it unless you allow it to. And if you die and the part of reality that is your consciousness will entirely seize to exist, so what? You won't be alive to give a dahm about it.

Acknowledge the truth, give it a polite nod, and continue on with your life. It will be there regardless, as will your immediate life

Yet who prohibits? Who prevents it from happening?

Eliezer seems absurdly optimistic to me. He is relying on some unseen entity to reach in and keep the laws of physics stable in our universe. We already see lots of evidence that they are not truly stable, for example we believe in both the electroweak transition and earlier transitions, of various natures depending on your school of physics. We /just saw/ in 1998 that unknown laws of physics can creep up and spring out by surprise, suddenly 'taking over' 74 percent of the Universe's postulated energ... (read more)

1[anonymous]6yOh come off it. There has to be some way that the universe actually works, at bottom. That way of working must be logical/causal, as if it's not, then everything happens and also nothing happens, including all logically contradictory things, more-or-less all the time. Since we don't observe anything remotely like that degree of sheer chaos, there must be laws. We don't always know the "absolute" laws, in fact we can only sometimes detect our ignorance (by having an experimental result we can't consistently explain), but we can build up models of universal lawfulness that keep working up to information leakage from an Outside universe (which would itself have to have laws of its own). There's no point worrying about Yog-Sothoth when we've already got Azathoth and Clippy on our plates.

On WWII - the "more or less the same way" is actually rather flexible. Usual advocates of "don't put it all on Hitler" say (AFAIK) something among the lines "historical balance was such that SOME major war involving Germany was bound to occur and, given the rise of tech, be about as deadly and as propaganda-fueled; Hitler did not invent or destroy so much tech as to change that particular statement" but not "there would arise a party against Jews which would win over Communists, make an alliance with another "co... (read more)

It’s interesting that you say that a Good God wouldn’t destroy a soul, as one of the biggest issues I’m currently finding myself having with Orthodox Judaism is that according to the Talmud at least, there have been a number of historical cases of souls being completely destroyed, which seems rather incompatible with the rest of Orthodox Jewish morality....I don’t know about the Christian or Muslim God, but they do both seem to believe that some people burn in hell forever, which is arguably worse than simply not existing. I really don’t get how this isn’t discussed more often in conventional theism...