ITYM 1. 100 people die, with certainty.
Care to test your skills against the Repugnant Conclusion? http://plato.stanford.edu/entries/repugnant-conclusion/
A life barely worth living is worth living. I see no pressing need to disagree with the Repugnant Conclusion itself.
However, I suspect there is a lot of confusion between "a life barely worth living" and "a life barely good enough that the person won't commit suicide".
A life barely good enough that the person won't commit suicide is well into the negatives.
Not to mention the confusion between "a life barely worth living" and "a life that has some typical number of bad experiences in it and barely any good experiences".
Whilst the your analysis of life-saving choices seems fairly uncontentious, I'm not entirely convinced that the arithmetic of different types of suffering add together the way you assume. It seems at least plausible to me that where dust motes are individual points, torture is a section of a contiuous line, and thus you can count the points, or you can measure the lengths of different lines, but no number of the former will add up to the latter.
A dust speck takes a finite time, not an instant. Unless I'm misunderstanding you, this makes them lines, not points.
I'm sorry, but I find this line of argument not very useful. If I remember correctly (which I may not be doing), a googolplex is larger than the estimated number of atoms in the universe. Nobody has any idea of what it implies except "really, really big", so when your concepts get up there, people have to do the math, since the numbers mean nothing. Most of us would agree that having a really really lot of people bothered just a bit is better than having one person suffer for a long life. That has little to do with math and a lot to do with o...
Nobody has any idea of what it implies except "really, really big", so when your concepts get up there, people have to do the math, since the numbers mean nothing.
This applies just as much to numbers such as million and billion, which people mixes up regularly; the problem though is that people dont do the math, despite not understanding the magnitudes if the numbers, and those numbers of people are actually around.
Personaly, if I first try to visualize a crowd of a hundred people, and then a crowd of a thousand, the second crowd seems about three times as large. If I start with a thousand, and then try a hundred, this time around the hundred people crowd seems a lot bigger than it did last time. And the bigger numbers I try with, the worse it gets, and there is a long way to go to get to 7'000'000'000(# of people of earth). All sorts of biases seems to be at work here, anchoring among them. Result: Shut up and multiply!
[Edit: Spelling]
One can easily make an argument like the torture vs. dust specks argument to show that the Repugnant Conclusion is not only not repugnant, but certainly true.
More intuitively, if it weren't true, we could find some population of 10,000 persons at some high standard of living, such that it would be morally praiseworthy to save their lives at the cost of a googolplex galaxies filled with intelligent beings. Most people would immediately say that this is false, and so the Repugnant Conclusion is true.
Eliezer, I am skeptical that sloganeerings ("shut up and calculate") will not get you across this philosophical chasm: Why do you define the best one-off choice as the choice that would be prefered over repeated trials?
Can someone please post a link to a paper on mathematics, philosophy, anything, that explains why there's this huge disconnect between "one-off choices" and "choices over repeated trials"? Lee?
Here's the way across the philosophical "chasm": write down the utility of the possible outcomes of your action. Use probability to find the expected utility. Do it for all your actions. Notice that if you have incoherent preferences, after a while, you expect your utility to be lower than if you do not have incoherent preferences.
You mi...
Consider these two facts about me:
(1) It is NOT CLEAR to me that saving 1 person with certainty is morally equivalent to saving 2 people when a fair coin lands heads in a one-off deal.
(2) It is CLEAR to me that saving 1000 people with p=.99 is morally better than saving 1 person with certainty.
Models are supposed to hew to the facts. Your model diverges from the facts of human moral judgments, and you respond by exhorting us to live up to your model.
Why should we do that?
Torture vs dust specks, let me see:
What would you choose for the next 50 days:
The consequence of choice 2 would be the death of one person.
Yudkowsky would choose 2, I would choose 1.
This is a question of threshold. Below certain thresholds things don't have much effect. So you cannot simply add up.
Another example:
What do you choose? Can we add up the discomfort caused by the one coin on each of 1,000,000 people?
These are simply false comparisons.
Had Eliezer talked about torturing someone through the use of googelplex of dust specks, your comparison might have merit, but as is it seems to be deliberately missing the point.
Certainly, speaking for someone else is often inappropriate, and in this case is simple strawmanning.
The comparison is invalid because the torture and dust specks are being compared as negatively-valued ends in themselves. We're comparing U(torture one person for 50 years) and U(dust speck one person) * 3^^^3. But you can't determine whether to take 1 ml of water per day from 100,000 people or 10 liters of water per day from 1 person by adding up the total amount of water in each case, because water isn't utility.
Perhaps this is just my misunderstanding of utility, but I think his point was this: I don't understand how adding up utility is obviously a legitimate thing to do, just like how you claim that adding up water denial is obviously not a legitimate thing to do. In fact, it seems to me as though the negative utility of getting a dust speck in the eye is comparable to the negative utility of being denied a milliliter of water, while the negative utility of being tortured for a lifetime is more or less equivalent to the negative utility of dying of thirst. I don't see why it is that the one addition is valid while the other isn't.
If this is just me misunderstanding utility, could you please point me to some readings so that I can better understand it?
Eliezer, can you explain what you mean by saying "it's the same gamble"? If the point is to compare two options and choose one, then what matters is their values relative to each other. So, 400 certain lives saved is better than a 90% chance of 500 lives saved and 10% chance of 500 deaths, which is itself better than 400 certain deaths.
Perhaps it would help to define the parameters more clearly. Do your first two options have an upper limit of 500 deaths (as the second two options seem to), or is there no limit to the number of deaths that may occur apart from the lucky 4-500?
Many were proud of this choice, and indignant that anyone should choose otherwise: "How dare you condone torture!" I don't think that's a fair characterization of that debate. A good number of people using many different reasons thought something along the lines of negligible "harm" * 3^^^3<50 years of torture. That many people spraining their ankle or something would be a different story. Those harms are different enough that it's by no means obvious which we should prefer, and it's not clear that trying to multiply is really productive, whereas your examples in this entry are indeed obvious.
"The primary thing is to help others, whatever the means. So shut up and multiply!"
Would you submit to torture for 50 years to save countless people? I'm not sure I would, but I think I'm more comfortable with the idea of being self-interested and seeing all things through the prism of self interest.
Similar problem: if you had this choice--you can die peacefully and experience no afterlife, or literally experience hell for 100 years if one was rewarded with an eternity of heaven, would you choose the latter? Calculating which provides the greatest utility, the latter would be preferable, but I'm not sure I would choose it.
Eliezer, as I'm sure you know, not everything can be put on a linear scale. Momentary eye irritation is not the same thing as torture. Momentary eye irritations should be negligible in the moral calculus, even when multiplied by googleplex^^^googleplex. 50 years of torture could break someone's mind and lead to their destruction. You're usually right on the mark, but not this time.
Would you pay one cent to prevent one googleplex of people from having a momentary eye irration?
Torture can be put on a money scale as well: many many countries use torture in war, but we don't spend huge amounts of money publicizing and shaming these people (which would reduce the amount of torture in the world).
In order to maximize the benefit of spending money, you must weigh sacred against unsacred.
To get back to the 'human life' examples EY quotes. Imagine instead the first scenario pair as being the last lifeboat on the Titanic. You can launch it safely with 40 people on board, or load in another 10 people, who would otherwise die a certain, wet, and icy death, and create a 1 in 10 chance that it will sink before the Carpathia arrives, killing all. I find that a strangely more convincing case for option 2. The scenarios as presented combine emotionally salient and abstract elements, with the result that the emotionally salient part will tend to be foreground, and the '% probabilities' as background. After all no-one ever saw anyone who was 10% dead (jokes apart).
Eliezer's point would have been valid, had he chosen almost anything other than momentary eye irritation. Even the momentary eye-irritation example would work if the eye irritation would lead to serious harm (e.g. eye inflammation and blindness) in a small proportion of those afflicted with the speck of dust. If the predicted outcome was millions of people going blind (and then you have to consider the resulting costs to society), then Eliezer is absolutely right: shut-up and do the math.
GreedyAlgorithm, this is the conversation I want to have.
The sentence in your argument that I cannot swallow is this one: "Notice that if you have incoherent preferences, after a while, you expect your utility to be lower than if you do not have incoherent preferences." This is circular, is it not?
You want to establish that any decision, x, should be made in accordance w/ maximum expected utility theory ("shut up and calculate"). You ask me to consider X = {x_i}, the set of many decisions over my life ("after a while"). You sa...
(I should say that I assumed that a bag of decisions is worth as much as the sum of the utilities of the individual decisions.)
I'm seconding the worries of people like the anonymous of the first comment and Wendy. I look at the first, and I think "with no marginal utility, it's an expected value of 400 vs an expected value of 450." I look at the second and think "with no marginal utility, it's an expected value of -400 vs. an expected value of -50." Marginal utility considerations--plausible if these are the last 500 people on Earth--sway the first case much more easily than they do the second case.
So we can keep doing this, gradually - very gradually - diminishing the degree of discomfort...
Eliezer, your readiness to assume that all 'bad things' are on a continuous scale, linear or no, really surprises me. Put your enormous numbers away, they're not what people are taking umbrage at. Do you think that if a googol doesn't convince us, perhaps a googolplex will? Or maybe 3^^^3? If x and y are finite, there will always be a quantity of x that exceeds y, and vice versa. We get the maths, we just don't agree that the phenomena are comparable. Broken ankle? Stubbing your toe? Possibly, there is certainly more of a tangible link there, but you're still imposing your judgment on how the mind experiences and deals with discomfort on us all and calling it rationality. It isn't.
Put simply - a dust mote registers exactly zero on my torture scale, and torture registers fundamentally off the scale (not just off the top, off) on my dust mote scale.
You're asking how many biscuits equal one steak, and then when one says 'there is no number', accusing him of scope insensitivity.
Well, he didn't actually identify dust mote disutility as zero; he says that dust motes register as zero on his torture scale. He goes on to mention that torture isn't on his dust-mote scale, so he isn't just using "torture scale" as a synonym for "disutility scale"; rather, he is emphasizing that there is more than just a single "(dis)utility scale" involved. I believe his contention is that the events (torture and dust-mote-in-the-eye) are fundamentally different in terms of "how the mind experiences and deals with [them]", such that no amount of dust motes can add up to the experience of torture... even if they (the motes) have a nonzero amount of disutility.
I believe I am making much the same distinction with my separation of disutility into trivial and non-trivial categories, where no amount of trivial disutility across multiple people can sum to the experience of non-trivial disutility. There is a fundamental gap in the scale (or different scales altogether, à la Jones), a difference in how different amounts of disutility work for humans. For a more concrete example of how this might work, suppose I steal one cent each from one billi...
Followup to: Torture vs. Dust Specks, Zut Allais, Rationality Quotes 4
Suppose that a disease, or a monster, or a war, or something, is killing people. And suppose you only have enough resources to implement one of the following two options:
Most people choose option 1. Which, I think, is foolish; because if you multiply 500 lives by 90% probability, you get an expected value of 450 lives, which exceeds the 400-life value of option 1. (Lives saved don't diminish in marginal utility, so this is an appropriate calculation.)
"What!" you cry, incensed. "How can you gamble with human lives? How can you think about numbers when so much is at stake? What if that 10% probability strikes, and everyone dies? So much for your damned logic! You're following your rationality off a cliff!"
Ah, but here's the interesting thing. If you present the options this way:
Then a majority choose option 2. Even though it's the same gamble. You see, just as a certainty of saving 400 lives seems to feel so much more comfortable than an unsure gain, so too, a certain loss feels worse than an uncertain one.
You can grandstand on the second description too: "How can you condemn 100 people to certain death when there's such a good chance you can save them? We'll all share the risk! Even if it was only a 75% chance of saving everyone, it would still be worth it - so long as there's a chance - everyone makes it, or no one does!"
You know what? This isn't about your feelings. A human life, with all its joys and all its pains, adding up over the course of decades, is worth far more than your brain's feelings of comfort or discomfort with a plan. Does computing the expected utility feel too cold-blooded for your taste? Well, that feeling isn't even a feather in the scales, when a life is at stake. Just shut up and multiply.
Previously on Overcoming Bias, I asked what was the least bad, bad thing that could happen, and suggested that it was getting a dust speck in your eye that irritated you for a fraction of a second, barely long enough to notice, before it got blinked away. And conversely, a very bad thing to happen, if not the worst thing, would be getting tortured for 50 years.
Now, would you rather that a googolplex people got dust specks in their eyes, or that one person was tortured for 50 years? I originally asked this question with a vastly larger number - an incomprehensible mathematical magnitude - but a googolplex works fine for this illustration.
Most people chose the dust specks over the torture. Many were proud of this choice, and indignant that anyone should choose otherwise: "How dare you condone torture!"
This matches research showing that there are "sacred values", like human lives, and "unsacred values", like money. When you try to trade off a sacred value against an unsacred value, subjects express great indignation (sometimes they want to punish the person who made the suggestion).
My favorite anecdote along these lines - though my books are packed at the moment, so no citation for now - comes from a team of researchers who evaluated the effectiveness of a certain project, calculating the cost per life saved, and recommended to the government that the project be implemented because it was cost-effective. The governmental agency rejected the report because, they said, you couldn't put a dollar value on human life. After rejecting the report, the agency decided not to implement the measure.
Trading off a sacred value (like refraining from torture) against an unsacred value (like dust specks) feels really awful. To merely multiply utilities would be too cold-blooded - it would be following rationality off a cliff...
But let me ask you this. Suppose you had to choose between one person being tortured for 50 years, and a googol people being tortured for 49 years, 364 days, 23 hours, 59 minutes and 59 seconds. You would choose one person being tortured for 50 years, I do presume; otherwise I give up on you.
And similarly, if you had to choose between a googol people tortured for 49.9999999 years, and a googol-squared people being tortured for 49.9999998 years, you would pick the former.
A googolplex is ten to the googolth power. That's a googol/100 factors of a googol. So we can keep doing this, gradually - very gradually - diminishing the degree of discomfort, and multiplying by a factor of a googol each time, until we choose between a googolplex people getting a dust speck in their eye, and a googolplex/googol people getting two dust specks in their eye.
If you find your preferences are circular here, that makes rather a mockery of moral grandstanding. If you drive from San Jose to San Francisco to Oakland to San Jose, over and over again, you may have fun driving, but you aren't going anywhere. Maybe you think it a great display of virtue to choose for a googolplex people to get dust specks rather than one person being tortured. But if you would also trade a googolplex people getting one dust speck for a googolplex/googol people getting two dust specks et cetera, you sure aren't helping anyone. Circular preferences may work for feeling noble, but not for feeding the hungry or healing the sick.
Altruism isn't the warm fuzzy feeling you get from being altruistic. If you're doing it for the spiritual benefit, that is nothing but selfishness. The primary thing is to help others, whatever the means. So shut up and multiply!
And if it seems to you that there is a fierceness to this maximization, like the bare sword of the law, or the burning of the sun - if it seems to you that at the center of this rationality there is a small cold flame -
Well, the other way might feel better inside you. But it wouldn't work.
And I say also this to you: That if you set aside your regret for all the spiritual satisfaction you could be having - if you wholeheartedly pursue the Way, without thinking that you are being cheated - if you give yourself over to rationality without holding back, you will find that rationality gives to you in return.
But that part only works if you don't go around saying to yourself, "It would feel better inside me if only I could be less rational."
Chimpanzees feel, but they don't multiply. Should you be sad that you have the opportunity to do better? You cannot attain your full potential if you regard your gift as a burden.
Added: If you'd still take the dust specks, see Unknown's comment on the problem with qualitative versus quantitative distinctions.